Nov 03 22:19:28.160268 ip-10-0-132-187 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 03 22:19:28.160283 ip-10-0-132-187 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 03 22:19:28.160293 ip-10-0-132-187 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 03 22:19:28.160656 ip-10-0-132-187 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 03 22:19:38.305670 ip-10-0-132-187 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 03 22:19:38.305690 ip-10-0-132-187 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9819344eaed44147a3939b34f03f4c0d --
Nov 03 22:21:43.470816 ip-10-0-132-187 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 22:21:43.812356 ip-10-0-132-187 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:21:43.812356 ip-10-0-132-187 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 22:21:43.812938 ip-10-0-132-187 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:21:43.812938 ip-10-0-132-187 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:21:43.815536 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815458    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815537    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815541    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815545    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815547    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815550    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815553    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815556    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815558    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815567    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815571    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815573    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815576    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:21:43.815574 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815579    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815582    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815585    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815587    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815590    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815592    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815595    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815597    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815600    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815603    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815605    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815608    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815610    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815613    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815616    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815618    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815621    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815623    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815626    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815628    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:21:43.815881 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815631    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815633    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815636    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815638    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815640    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815643    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815645    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815648    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815651    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815653    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815656    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815659    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815661    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815664    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815667    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815669    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815672    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815674    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815676    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:21:43.816364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815679    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815681    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815684    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815686    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815689    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815691    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815694    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815697    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815701    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815706    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815709    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815712    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815715    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815717    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815720    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815724    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815728    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815731    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:21:43.816813 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815734    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815737    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815740    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815742    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815746    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815748    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815752    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815755    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815758    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815761    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815764    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815766    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815770    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815772    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815775    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815778    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815780    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815783    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815785    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815788    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:21:43.817260 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815790    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815793    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815795    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815798    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815800    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815803    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815805    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815807    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815810    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815812    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815815    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815817    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815819    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815822    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815825    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.815827    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816621    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816628    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816632    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816636    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:21:43.817714 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816639    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816642    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816645    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816647    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816650    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816652    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816655    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816657    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816660    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816663    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816665    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816668    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816670    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816673    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816675    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816678    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816681    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816683    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816685    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:21:43.818236 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816688    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816690    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816693    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816696    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816698    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816700    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816703    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816705    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816707    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816710    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816712    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816716    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816720    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816724    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816727    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816730    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816732    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816735    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816737    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816740    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:21:43.818703 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816742    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816745    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816747    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816750    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816752    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816754    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816759    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816763    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816766    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816768    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816771    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816773    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816775    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816778    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816780    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816783    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816785    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816787    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816790    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:21:43.819193 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816792    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816795    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816797    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816800    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816803    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816806    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816808    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816812    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816815    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816818    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816821    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816823    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816826    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816829    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816831    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816834    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816836    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816839    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816841    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816844    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:21:43.819709 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816847    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816849    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816852    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816854    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816857    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816860    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816862    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816864    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816867    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816869    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816872    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816874    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816876    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816879    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816881    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816884    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816888    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816891    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816894    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:21:43.820312 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816897    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816899    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816902    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816904    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.816908    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817456    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817463    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817467    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817470    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817473    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817475    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817478    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817480    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817483    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817486    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817488    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817491    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817493    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817496    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817498    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:21:43.820919 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817501    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817503    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817506    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817508    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817510    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817513    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817515    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817518    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817520    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817523    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817526    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817529    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817532    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817534    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817537    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817539    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817542    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817544    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817547    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817549    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:21:43.821421 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817552    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817554    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817556    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817559    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817561    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817564    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817566    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817569    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817572    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817574    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817577    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817579    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817581    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817584    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817586    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817590    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817594    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817601    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817606    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:21:43.821896 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817610    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817614    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817618    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817621    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817624    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817628    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817631    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817634    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817637    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817639    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817642    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817644    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817646    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817649    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817651    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817654    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817656    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817658    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817661    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:21:43.822364 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817663    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817665    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817668    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817670    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817673    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817675    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817677    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817680    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817682    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817685    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817687    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817689    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817692    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817694    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817696    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817699    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817702    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817704    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817709    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817712    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:21:43.822820 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817715    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817717    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817720    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817722    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817725    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817728    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817731    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817733    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817736    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817738    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817741    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817744    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.817747    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.819407    2576 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.819420    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.819500    2576 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.819509    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820195    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820206    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820214    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820222    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 22:21:43.823406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820227    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820239    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820245    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820250    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820255    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820261    2576 flags.go:64] FLAG: --cgroup-root=""
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820265    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820270    2576 flags.go:64] FLAG: --client-ca-file=""
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820275    2576 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820281    2576 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820293    2576 flags.go:64] FLAG: --cluster-domain=""
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820313    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820319    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820324    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820330    2576 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820337    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820342    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820347    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820356    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820361    2576 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820366    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820371    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820376    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820381    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 22:21:43.823903 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820388    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820394    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820398    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820408    2576 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820413    2576 flags.go:64] FLAG: --enable-server="true"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820418    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820425    2576 flags.go:64] FLAG: --event-burst="100"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820430    2576 flags.go:64] FLAG: --event-qps="50"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820435    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820440    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820445    2576 flags.go:64] FLAG: --eviction-hard=""
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820456    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820461    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820466    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820471    2576 flags.go:64] FLAG: --eviction-soft=""
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820476    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820480    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820485    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820490    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820495    2576 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820504    2576 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820509    2576 flags.go:64] FLAG: --feature-gates=""
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820515    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820520    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820525    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 22:21:43.824529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820530    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820535    2576 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820540    2576 flags.go:64] FLAG: --help="false"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820549    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820555    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820560    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820565    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820571    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820577    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820582    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820586    2576 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820591    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820601    2576 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820606    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820611    2576 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820615    2576 flags.go:64] FLAG: --kube-reserved=""
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820620    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820783    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820792    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820797    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820801    2576 flags.go:64] FLAG: --lock-file=""
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820804    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820808    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820813    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 22:21:43.825144 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820827    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820831    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820835    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820838    2576 flags.go:64] FLAG: --logging-format="text"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820842    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820845    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820848    2576 flags.go:64] FLAG: --manifest-url=""
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820851    2576 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820857    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820860    2576 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820865    2576 flags.go:64] FLAG: --max-pods="110"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820868    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820872    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820874    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820877    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820883    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820886    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820889    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820899    2576 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820902    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820905    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820909    2576 flags.go:64] FLAG: --pod-cidr=""
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820911    2576 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820914    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820917    2576 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 22:21:43.825699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820921    2576 flags.go:64] FLAG: --port="10250"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820924    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820927    2576 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-06f62ae6ad0a33845"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820931    2576 flags.go:64] FLAG: --qos-reserved=""
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820934    2576 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820937    2576 flags.go:64] FLAG: --register-node="true"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820940    2576 flags.go:64] FLAG: --register-with-taints=""
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820944    2576 flags.go:64] FLAG: --registry-burst="10"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820947    2576 flags.go:64] FLAG: --registry-qps="5"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820950    2576 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820953    2576 flags.go:64] FLAG: --reserved-memory=""
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820957    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820960    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820963    2576 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820966    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820969    2576 flags.go:64] FLAG: --runonce="false"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820972    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820975    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820979    2576 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820981    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820984    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820987    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820990    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820993    2576 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820996    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.820999    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 22:21:43.826298 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821002    2576 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821005    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821008    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821010    2576 flags.go:64] FLAG: --system-cgroups=""
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821013    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821020    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821024    2576 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821027    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821033    2576 flags.go:64] FLAG: --tls-min-version=""
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821036    2576 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821038    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821041    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821045    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821048    2576 flags.go:64] FLAG: --v="2"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821052    2576 flags.go:64] FLAG: --version="false"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821064    2576 flags.go:64] FLAG: --vmodule=""
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821068    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821071    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821254    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821258    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821263    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821267    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821270    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:21:43.826906 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821274    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821276    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821279    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821281    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821284    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821286    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821289    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821291    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821294    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821296    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821299    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821301    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821304    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821306    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821309    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821314    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821317    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821319    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821322    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821324    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:21:43.827772 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821327    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821329    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821332    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821334    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821337    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821339    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821341    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821344    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821346    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821350    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821353    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821355    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821358    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821360    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821363    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821365    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821368    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821370    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821373    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:21:43.828636 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821375    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821378    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821380    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821383    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821385    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821388    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821390    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821393    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821399    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821401    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821404    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821406    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821409    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821411    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821414    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821416    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821419    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821422    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821426    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:21:43.829466 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821430    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821433    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821437    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821440    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821443    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821446    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821449    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821451    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821454    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821456    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821458    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821461    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821463    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821466    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821469    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821471    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821473    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821476    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821478    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821480    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:21:43.830030 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821483    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821487    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821490    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821492    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821495    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821498    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821500    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821503    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821506    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821509    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821511    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821514    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821517    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821519    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821522    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821524    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821527    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821530    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821532    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:21:43.830854 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821535    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821537    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821540    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821542    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.821568    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821603    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821608    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821611    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821614    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821617    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821619    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821622    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821624    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821627    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821630    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:21:43.831700 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821635    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821640    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821643    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821646    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821649    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821651    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821654    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821656    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821659    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821661    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821664    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821666    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821669    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821671    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821674    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821676    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821679    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821682    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821684    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821687    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:21:43.832384 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821689    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821692    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821694    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821697    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821699    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821701    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821704    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821706    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821709    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821711    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821714    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821716    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821720    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821723    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821725    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821728    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821730    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821733    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821735    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821738    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:21:43.832891 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821740    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821743    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821745    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821748    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821750    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821753    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821755    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821758    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821760    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821764    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821766    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821769    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821771    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821773    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821776    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821778    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821781    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821784    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821786    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:21:43.833710 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821789    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821793    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821796    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821798    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821801    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821805    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821807    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821810    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821812    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821815    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821818    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821820    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821822    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821825    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821827    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821830    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821832    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821835    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821837    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821839    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:21:43.834508 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821842    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821844    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821847    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821849    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821852    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821854    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821857    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821859    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821862    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821864    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821868    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821871    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821873    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821875    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821878    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821880    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.821884    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.829634    2576 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.829659    2576 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 22:21:43.835007 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.830231    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.830246    2576 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830340    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830350    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830357    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830362    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830366    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830370    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830375    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830379    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830383    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830387    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830391    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830396    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830400    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830404    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830409    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830413    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830417    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:21:43.835632 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830421    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830425    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830431    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830438    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830443    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830448    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830452    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830457    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830461    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830466    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830470    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830474    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830480    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830485    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830492    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830497    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830502    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830506    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830511    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:21:43.836148 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830515    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830519    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830523    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830527    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830532    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830536    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830540    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830544    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830548    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830552    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830556    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830561    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830565    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830569    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830574    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830578    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830582    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830586    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830590    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:21:43.836674 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830594    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830598    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830602    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830606    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830611    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830615    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830620    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830625    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830629    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830633    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830637    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830641    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830645    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830649    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830653    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830658    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830661    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830665    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830669    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830674    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:21:43.837335 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830678    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830681    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830686    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830691    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830696    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830700    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830704    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830708    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830712    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830716    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830720    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830725    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830729    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830733    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830737    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830743    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830747    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830751    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830756    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:21:43.838197 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830761    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830766    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830770    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830774    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830778    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830782    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830787    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830791    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830796    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830800    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830804    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.830808    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.830849    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831094    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831103    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831109    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:21:43.838963 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831113    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831118    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831122    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831127    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831131    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831136    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831140    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831145    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831149    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831153    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831158    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831162    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831166    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831170    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831174    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831178    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831184    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831188    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831193    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:21:43.839642 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831198    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831201    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831206    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831210    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831214    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831218    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831222    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831226    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831231    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831235    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831240    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831244    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831249    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831253    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831258    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831262    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831267    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831271    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831275    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:21:43.840184 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831280    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831285    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831289    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831293    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831297    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831304    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831309    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831314    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831321    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831328    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831332    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831337    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831343    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831348    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831353    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831357    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831361    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831365    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831369    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:21:43.840639 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831373    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831377    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831382    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831386    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831390    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831395    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831399    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831403    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831407    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831411    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831415    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831420    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831424    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831429    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831433    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831437    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831441    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831445    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831449    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831453    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:21:43.841116 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831457    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831462    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831466    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831471    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831475    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831479    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831483    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831489    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831493    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831497    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831502    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831506    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831510    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831514    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831518    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831523    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831527    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831531    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831535    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831540    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:21:43.841624 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831544    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831548    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831553    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831557    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831561    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831566    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.831606    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831686    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831693    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831698    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831702    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831706    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831710    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831714    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831718    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831722    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:21:43.842266 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831726    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831731    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831735    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831740    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831745    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831749    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831753    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831757    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831762    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831766    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831771    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831775    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831781    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831785    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831789    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831793    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831799    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831805    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831811    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:21:43.842743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831817    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831824    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831828    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831834    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831839    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831843    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831847    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831851    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831855    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831859    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831863    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831868    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831872    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831876    2576 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831881    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831885    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831889    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831893    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831898    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831902    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:21:43.843218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831907    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831911    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831915    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831919    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831923    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831928    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831932    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831936    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831940    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831944    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831948    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831953    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831958    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831962    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831966    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831971    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831975    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831979    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831983    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831987    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:21:43.843731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831991    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.831995    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832000    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832004    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832008    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832012    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832016    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832020    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832025    2576 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832032    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832036    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832045    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832050    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832054    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832058    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832062    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832066    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832070    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832093    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832097    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:21:43.844218 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832101    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832104    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832108    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832112    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832117    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832121    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832125    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832129    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832134    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832138    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832142    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832146    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832150    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832154    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832158    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832162    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832167    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:43.832172    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.832266    2576 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 22:21:43.844743 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.835167    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 03 22:21:43.845793 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.836028    2576 server.go:1014] "Starting client certificate rotation"
Nov 03 22:21:43.845793 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.836111    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 22:21:43.845793 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.836151    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 22:21:43.845793 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.837889    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 22:21:43.845793 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.839735    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 22:21:43.849546 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.849526    2576 log.go:25] "Validated CRI v1 runtime API"
Nov 03 22:21:43.852570 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.852552    2576 log.go:25] "Validated CRI v1 image API"
Nov 03 22:21:43.853464 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.853430    2576 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 22:21:43.857710 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.857690    2576 fs.go:135] Filesystem UUIDs: map[761f7cf9-0f4b-4f70-b121-df7a570b9e3e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 98c4c6fe-bd7d-470e-a46f-2345f99e7201:/dev/nvme0n1p4]
Nov 03 22:21:43.857774 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.857711    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 03 22:21:43.862905 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.862784    2576 manager.go:217] Machine: {Timestamp:2026-11-03 22:21:43.861457971 +0000 UTC m=+0.284301492 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100178 MemoryCapacity:32812003328 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ceb707cbc1d7c7df0dbc3ea47a0fe SystemUUID:ec2ceb70-7cbc-1d7c-7df0-dbc3ea47a0fe BootID:9819344e-aed4-4147-a393-9b34f03f4c0d Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16405999616 Type:vfs Inodes:4005371 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:79:fd:a5:68:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:79:fd:a5:68:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:a1:b2:5b:e2:8e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812003328 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 22:21:43.863443 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.863432    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 22:21:43.863540 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.863527    2576 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 22:21:43.866133 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.866103    2576 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 22:21:43.866287 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.866136    2576 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-187.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 22:21:43.866353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.866289    2576 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 22:21:43.866353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.866298    2576 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 22:21:43.866353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.866332    2576 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 22:21:43.866353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.866347    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 22:21:43.867336 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.867325    2576 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 22:21:43.870110 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.870096    2576 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 22:21:43.870226 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.870217    2576 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 22:21:43.870836 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.870827    2576 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 22:21:43.870870 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.870840    2576 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 22:21:43.870870 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.870852    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 22:21:43.870870 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.870861    2576 kubelet.go:405] "Adding apiserver pod source"
Nov 03 22:21:43.870870 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.870869    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 22:21:43.871292 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.871281    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 22:21:43.873236 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.873216    2576 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 22:21:43.879251 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.879237    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 22:21:43.880039 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880026    2576 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 22:21:43.880103 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880058    2576 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 22:21:43.880247 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880236    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 22:21:43.880275 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880253    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 22:21:43.880275 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880259    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 22:21:43.880275 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880265    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 22:21:43.880275 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880273    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 22:21:43.880379 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880283    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 22:21:43.880752 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880744    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 22:21:43.880780 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880753    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 22:21:43.880780 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880766    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 22:21:43.880780 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880772    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 22:21:43.880856 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880784    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 22:21:43.880856 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.880794    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 22:21:43.881354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.881345    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 22:21:43.881392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.881357    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 22:21:43.885464 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.885449    2576 server.go:1263] "Started kubelet"
Nov 03 22:21:43.885933 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.885518    2576 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 22:21:43.886037 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.885992    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 22:21:43.886091 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.886059    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 22:21:43.886337 ip-10-0-132-187 systemd[1]: Started Kubernetes Kubelet.
Nov 03 22:21:43.887282 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.887264    2576 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 22:21:43.887808 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.887792    2576 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 22:21:43.891740 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:43.891718    2576 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 03 22:21:43.893194 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.893173    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 22:21:43.893721 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.893703    2576 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 22:21:43.893822 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:43.893803    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-187.us-west-1.compute.internal\" not found"
Nov 03 22:21:43.894423 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.894397    2576 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 22:21:43.894423 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.894418    2576 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 22:21:43.894423 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.894422    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 22:21:43.894628 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.894460    2576 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 22:21:43.894628 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.894465    2576 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 22:21:43.894924 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.894904    2576 factory.go:55] Registering systemd factory
Nov 03 22:21:43.894993 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.894942    2576 factory.go:223] Registration of the systemd container factory successfully
Nov 03 22:21:43.895219 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.895206    2576 factory.go:153] Registering CRI-O factory
Nov 03 22:21:43.895274 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.895222    2576 factory.go:223] Registration of the crio container factory successfully
Nov 03 22:21:43.895329 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.895281    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 22:21:43.895329 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.895302    2576 factory.go:103] Registering Raw factory
Nov 03 22:21:43.895329 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.895312    2576 manager.go:1199] Started watching for new ooms in manager
Nov 03 22:21:43.896213 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.896201    2576 manager.go:319] Starting recovery of all containers
Nov 03 22:21:43.904240 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.904186    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 22:21:43.906981 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.906960    2576 manager.go:324] Recovery completed
Nov 03 22:21:43.912503 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.912488    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:21:43.915148 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.914559    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:21:43.915148 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.914603    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:21:43.915148 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.914618    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:21:43.915497 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.915483    2576 cpu_manager.go:226] "Starting" policy="none"
Nov 03 22:21:43.915555 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.915498    2576 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 22:21:43.915555 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.915486    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:21:43.915555 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.915519    2576 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 22:21:43.916608 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.916594    2576 policy_none.go:50] "Start"
Nov 03 22:21:43.916706 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.916612    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 22:21:43.916706 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.916631    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 22:21:43.916801 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.916709    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:21:43.916801 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.916746    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:21:43.916801 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.916762    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:21:43.917561 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.917549    2576 policy_none.go:44] "Start"
Nov 03 22:21:43.953699 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.953664    2576 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 22:21:43.955953 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.955934    2576 manager.go:341] "Starting Device Plugin manager"
Nov 03 22:21:43.956067 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:43.955969    2576 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 22:21:43.956067 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.955981    2576 server.go:87] "Starting device plugin registration server"
Nov 03 22:21:43.956257 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.956243    2576 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 22:21:43.956312 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.956257    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 22:21:43.956363 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.956344    2576 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 22:21:43.956450 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.956435    2576 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 22:21:43.956450 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:43.956450    2576 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 22:21:43.956917 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:43.956897    2576 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 22:21:43.956991 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:43.956939    2576 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-187.us-west-1.compute.internal\" not found"
Nov 03 22:21:44.032047 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.031893    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 22:21:44.032047 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.032046    2576 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 22:21:44.033965 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.032068    2576 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 22:21:44.033965 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.032119    2576 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 03 22:21:44.056352 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.056332    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:21:44.057209 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.057186    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:21:44.057270 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.057227    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:21:44.057270 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.057241    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:21:44.057270 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.057265    2576 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.133087 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.132984    2576 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"]
Nov 03 22:21:44.133194 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.133088    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:21:44.134108 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.134092    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:21:44.134165 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.134126    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:21:44.134165 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.134137    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:21:44.136251 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136240    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:21:44.136432 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.136465 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136455    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:21:44.136941 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136924    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:21:44.137006 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136949    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:21:44.137006 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136958    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:21:44.137006 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136981    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:21:44.137006 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136990    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:21:44.137166 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.136963    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:21:44.139003 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.138977    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.139003 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.139001    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:21:44.139726 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.139709    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:21:44.139796 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.139741    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:21:44.139796 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.139756    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:21:44.157218 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.157199    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-187.us-west-1.compute.internal\" not found" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.159886 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.159861    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-187.us-west-1.compute.internal\" not found" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.159964 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.159949    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 22:21:44.160034 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.160022    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kjv95"
Nov 03 22:21:44.192587 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.192556    2576 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-187.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 03 22:21:44.192720 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.192589    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-187.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 22:21:44.193202 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.192565    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-187.us-west-1.compute.internal.18e4a91f63e3c2f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-187.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-187.us-west-1.compute.internal,},FirstTimestamp:2026-11-03 22:21:43.88525132 +0000 UTC m=+0.308094839,LastTimestamp:2026-11-03 22:21:43.88525132 +0000 UTC m=+0.308094839,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-187.us-west-1.compute.internal,}"
Nov 03 22:21:44.295840 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.295799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cc67bfe6e75d748c4abbd6f6df49a40-config\") pod \"kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"9cc67bfe6e75d748c4abbd6f6df49a40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.295840 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.295841    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.296016 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.295864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.351165 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.351137    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:21:44.351319 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.351137    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:21:44.353120 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.353067    2576 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:21:44.396980 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.396921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cc67bfe6e75d748c4abbd6f6df49a40-config\") pod \"kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"9cc67bfe6e75d748c4abbd6f6df49a40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.396980 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.396953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.397152 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.396994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.397152 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.397038    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cc67bfe6e75d748c4abbd6f6df49a40-config\") pod \"kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"9cc67bfe6e75d748c4abbd6f6df49a40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.397152 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.397038    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.397152 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.397038    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.450274 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.450245    2576 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-132-187.us-west-1.compute.internal\" not found" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.451440 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.451424    2576 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:21:44.460447 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.460430    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.463949 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.463930    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.501562 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.501546    2576 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-187.us-west-1.compute.internal\" not found" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.650814 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.650734    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:21:44.651817 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.651800    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:21:44.651886 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.651835    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:21:44.651886 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.651848    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:21:44.651886 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.651874    2576 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.710275 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.710245    2576 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.710414 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.710286    2576 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-187.us-west-1.compute.internal\": node \"ip-10-0-132-187.us-west-1.compute.internal\" not found"
Nov 03 22:21:44.794358 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.794322    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.836285 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.836262    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 03 22:21:44.836743 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.836369    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 22:21:44.836743 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.836417    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 22:21:44.836743 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.836464    2576 kubelet.go:3370] "Failed creating a mirror pod" err="Post \"https://a6c0d62e96a584816a3ee7bc88cce2ee-6000754553a9ad3e.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.132.187:40372->3.131.233.76:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.836743 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.836501    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:21:44.836743 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:44.836457    2576 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a6c0d62e96a584816a3ee7bc88cce2ee-6000754553a9ad3e.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-132-187.us-west-1.compute.internal.18e4a91f65a36f94\": read tcp 10.0.132.187:40372->3.131.233.76:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-132-187.us-west-1.compute.internal.18e4a91f65a36f94  default   5277 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-187.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-187.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-187.us-west-1.compute.internal,},FirstTimestamp:2026-11-03 22:21:43 +0000 UTC,LastTimestamp:2026-11-03 22:21:44.057210189 +0000 UTC m=+0.480053708,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-187.us-west-1.compute.internal,}"
Nov 03 22:21:44.836743 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.836471    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 22:21:44.871292 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.871272    2576 apiserver.go:52] "Watching apiserver"
Nov 03 22:21:44.894099 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:44.894070    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 03 22:21:45.046029 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.045941    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:21:45.146712 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.146684    2576 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 22:21:45.147009 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.146989    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-84hzv","openshift-network-operator/iptables-alerter-lqjh7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-8ck4c","openshift-multus/multus-rkgdk","openshift-network-diagnostics/network-check-target-g9t4g","kube-system/konnectivity-agent-qj6mk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29","openshift-cluster-node-tuning-operator/tuned-2lfgw","openshift-dns/node-resolver-pvgnl","openshift-multus/network-metrics-daemon-vr2lh","openshift-ovn-kubernetes/ovnkube-node-d7l9z","kube-system/global-pull-secret-syncer-zhsp8","kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"]
Nov 03 22:21:45.149397 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.149381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:45.151369 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.151351    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.151478 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.151430    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:21:45.153296 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.153281    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.155261 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.155246    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.157269 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.157248    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.159312 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.159300    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:45.159373 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.159355    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:21:45.161525 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.161506    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.163559 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.163543    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.165530 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.165513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.165983 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.165958    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 22:16:45 +0000 UTC" deadline="2028-05-15 04:30:48.67242201 +0000 UTC"
Nov 03 22:21:45.165983 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.165983    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13398h9m3.506442047s"
Nov 03 22:21:45.168505 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.168491    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.170657 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.170640    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:45.170735 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.170692    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:21:45.173211 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.173196    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.195945 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.195928    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 22:21:45.200464 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-conf\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.200509 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-k8s-cni-cncf-io\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.200509 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.200577 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-hostroot\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.200577 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200534    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsjsb\" (UniqueName: \"kubernetes.io/projected/1f6b6450-cd73-4506-85d1-94ef030fe583-kube-api-access-zsjsb\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.200577 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-multus\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.200577 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-kubelet\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.200693 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-etc-kubernetes\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.200693 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cnibin\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.200693 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200644    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.200693 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-lib-modules\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.200818 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d457444-8760-482f-a08d-7b20024db9fa-tmp-dir\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.200818 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.200818 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-netd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.200818 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-slash\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.200818 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-netns\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.200976 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-etc-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.200976 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-tuned\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.200976 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77hn5\" (UniqueName: \"kubernetes.io/projected/ce2518fb-0bf8-4628-9927-c46800cb2b3c-kube-api-access-77hn5\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.200976 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200918    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-systemd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.200976 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-config\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.200976 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.200983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9zhc\" (UniqueName: \"kubernetes.io/projected/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-kube-api-access-j9zhc\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201046    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201101    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-var-lib-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-registration-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-run\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201150    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b0deac8-c535-4d85-b27e-968b22b55c14-agent-certs\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-dbus\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.201206 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn7q4\" (UniqueName: \"kubernetes.io/projected/8308da23-cdc6-4e50-91a2-9b158e17ca3b-kube-api-access-cn7q4\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-socket-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201249    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysconfig\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s49km\" (UniqueName: \"kubernetes.io/projected/6d457444-8760-482f-a08d-7b20024db9fa-kube-api-access-s49km\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-system-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201320    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ce2518fb-0bf8-4628-9927-c46800cb2b3c-serviceca\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-kubernetes\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz2b8\" (UniqueName: \"kubernetes.io/projected/8afabbf8-4122-42aa-95c6-75e6d41bf40d-kube-api-access-mz2b8\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201382    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-kubelet-config\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-netns\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-multus-certs\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.201479 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-ovn\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201501    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-node-log\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-env-overrides\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-modprobe-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-host\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201592    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-cni-binary-copy\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201610    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201638    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-tmp\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgckl\" (UniqueName: \"kubernetes.io/projected/ad0e377c-18f5-45ee-aabf-8b228f379db0-kube-api-access-wgckl\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-systemd\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201700    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlvlb\" (UniqueName: \"kubernetes.io/projected/dfd15a52-52ae-4718-a562-ba25778c39ea-kube-api-access-xlvlb\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201740    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-bin\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201754    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovn-node-metrics-cert\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2518fb-0bf8-4628-9927-c46800cb2b3c-host\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f6b6450-cd73-4506-85d1-94ef030fe583-iptables-alerter-script\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.201812 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f6b6450-cd73-4506-85d1-94ef030fe583-host-slash\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-system-cni-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-kubelet\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-systemd-units\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-log-socket\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-device-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.201994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmrdn\" (UniqueName: \"kubernetes.io/projected/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kube-api-access-qmrdn\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-var-lib-kubelet\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d457444-8760-482f-a08d-7b20024db9fa-hosts-file\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-sys-fs\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-sys\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b0deac8-c535-4d85-b27e-968b22b55c14-konnectivity-ca\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:45.202353 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202241    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-socket-dir-parent\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.202805 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-conf-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.202805 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202290    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.202805 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202307    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-cnibin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.202805 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202412    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-bin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.202805 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-daemon-config\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.202805 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.202805 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.202487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-script-lib\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.215701 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.215678    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 22:21:45.215804 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.215737    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sbg2z"
Nov 03 22:21:45.219313 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.219295    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 22:21:45.220123 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.220110    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 22:21:45.220223 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.220208    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xglhz\""
Nov 03 22:21:45.222214 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.222190    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 22:21:45.222214 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.222193    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 22:21:45.222370 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.222196    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 22:21:45.222564 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.222550    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 22:21:45.222798 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.222783    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 22:21:45.223197 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223185    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:21:45.223197 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223191    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 22:21:45.223311 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223257    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 22:21:45.223311 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223275    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 22:21:45.223407 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223348    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 22:21:45.223458 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223434    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zhswt\""
Nov 03 22:21:45.223510 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223483    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-dx5sz\""
Nov 03 22:21:45.223554 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223543    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q82l2\""
Nov 03 22:21:45.223611 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.223600    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dbbtb\""
Nov 03 22:21:45.225441 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.225430    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 22:21:45.225612 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.225601    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 22:21:45.242807 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.242789    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 22:21:45.253142 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.253120    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 22:21:45.253142 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.253137    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 22:21:45.253269 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.253159    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b2scz\""
Nov 03 22:21:45.253269 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.253158    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:21:45.255755 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.255737    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-px8pz\""
Nov 03 22:21:45.255801 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.255766    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 22:21:45.265769 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.265748    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 22:21:45.265854 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.265769    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 22:21:45.265854 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.265798    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 22:21:45.265854 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.265833    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 22:21:45.265949 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.265856    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 22:21:45.265949 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.265877    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mx976\""
Nov 03 22:21:45.269571 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.269448    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 22:21:45.269571 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.269473    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 22:21:45.269571 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.269485    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m2wh6\""
Nov 03 22:21:45.269571 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.269495    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 22:21:45.303404 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz2b8\" (UniqueName: \"kubernetes.io/projected/8afabbf8-4122-42aa-95c6-75e6d41bf40d-kube-api-access-mz2b8\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.303497 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303433    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-kubelet-config\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.303497 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-netns\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.303497 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303484    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-multus-certs\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-ovn\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-node-log\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-env-overrides\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303565    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-netns\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-modprobe-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303578    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-multus-certs\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-host\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303604    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-kubelet-config\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303584    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-ovn\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-cni-binary-copy\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.303646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303631    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-node-log\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303655    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-host\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-tmp\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303712    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-modprobe-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303724    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgckl\" (UniqueName: \"kubernetes.io/projected/ad0e377c-18f5-45ee-aabf-8b228f379db0-kube-api-access-wgckl\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-systemd\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlvlb\" (UniqueName: \"kubernetes.io/projected/dfd15a52-52ae-4718-a562-ba25778c39ea-kube-api-access-xlvlb\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303796    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-bin\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovn-node-metrics-cert\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2518fb-0bf8-4628-9927-c46800cb2b3c-host\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f6b6450-cd73-4506-85d1-94ef030fe583-iptables-alerter-script\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303897    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-systemd\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f6b6450-cd73-4506-85d1-94ef030fe583-host-slash\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-system-cni-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303950    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.304191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303958    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-bin\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.303993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-kubelet\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304001    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2518fb-0bf8-4628-9927-c46800cb2b3c-host\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-systemd-units\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304031    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-system-cni-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-log-socket\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304058    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f6b6450-cd73-4506-85d1-94ef030fe583-host-slash\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-device-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304044    2576 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmrdn\" (UniqueName: \"kubernetes.io/projected/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kube-api-access-qmrdn\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304163    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-var-lib-kubelet\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304176    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-kubelet\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304177    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-env-overrides\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d457444-8760-482f-a08d-7b20024db9fa-hosts-file\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-sys-fs\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304269    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-cni-binary-copy\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.305052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304281    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-device-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304329    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-systemd-units\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304318    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304374    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-sys-fs\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304379    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-log-socket\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304423    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-sys\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b0deac8-c535-4d85-b27e-968b22b55c14-konnectivity-ca\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304503    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-socket-dir-parent\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-conf-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-cnibin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-bin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-daemon-config\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.305936 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-script-lib\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-conf\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-k8s-cni-cncf-io\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-hostroot\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304753    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsjsb\" (UniqueName: \"kubernetes.io/projected/1f6b6450-cd73-4506-85d1-94ef030fe583-kube-api-access-zsjsb\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304790    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f6b6450-cd73-4506-85d1-94ef030fe583-iptables-alerter-script\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-multus\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304830    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-multus\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-kubelet\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-etc-kubernetes\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cnibin\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304912    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304934    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-lib-modules\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304946    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-bin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.306982 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304964    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d457444-8760-482f-a08d-7b20024db9fa-tmp-dir\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304987    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-etc-kubernetes\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304991    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.304998    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305016    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-kubelet\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305020    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-k8s-cni-cncf-io\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305019    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cnibin\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.305094    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305111    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305105    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-socket-dir-parent\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305136    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305174    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-hostroot\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305200    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305266    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-lib-modules\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.305300    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret podName:fd5c8298-1f37-4c24-bb52-1536b0e93028 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:45.805144825 +0000 UTC m=+2.227988351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret") pod "global-pull-secret-syncer-zhsp8" (UID: "fd5c8298-1f37-4c24-bb52-1536b0e93028") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305374    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-conf\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305401    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-cnibin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.307863 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305429    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-var-lib-kubelet\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305469    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-script-lib\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305495    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-sys\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-netd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305517    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-netd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305534    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-slash\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-netns\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305581    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d457444-8760-482f-a08d-7b20024db9fa-hosts-file\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305605    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-slash\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305608    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-etc-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-etc-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305630    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-netns\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-tuned\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305668    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77hn5\" (UniqueName: \"kubernetes.io/projected/ce2518fb-0bf8-4628-9927-c46800cb2b3c-kube-api-access-77hn5\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305676    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-conf-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-systemd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305685    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d457444-8760-482f-a08d-7b20024db9fa-tmp-dir\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-config\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305774    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-systemd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305839    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305667    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-daemon-config\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9zhc\" (UniqueName: \"kubernetes.io/projected/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-kube-api-access-j9zhc\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-var-lib-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305973    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b0deac8-c535-4d85-b27e-968b22b55c14-konnectivity-ca\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.305996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-registration-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306040    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-registration-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.306043    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-run\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.306120    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs podName:ad0e377c-18f5-45ee-aabf-8b228f379db0 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:45.806104804 +0000 UTC m=+2.228948323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs") pod "network-metrics-daemon-vr2lh" (UID: "ad0e377c-18f5-45ee-aabf-8b228f379db0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306141    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-run\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306151    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-var-lib-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b0deac8-c535-4d85-b27e-968b22b55c14-agent-certs\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:45.308880 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-dbus\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn7q4\" (UniqueName: \"kubernetes.io/projected/8308da23-cdc6-4e50-91a2-9b158e17ca3b-kube-api-access-cn7q4\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306269    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-socket-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306336    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-dbus\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysconfig\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s49km\" (UniqueName: \"kubernetes.io/projected/6d457444-8760-482f-a08d-7b20024db9fa-kube-api-access-s49km\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-system-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ce2518fb-0bf8-4628-9927-c46800cb2b3c-serviceca\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-kubernetes\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306562    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-kubernetes\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306585    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-config\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306607    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-system-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306626    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-socket-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.306671    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysconfig\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.307593    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ce2518fb-0bf8-4628-9927-c46800cb2b3c-serviceca\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.307759    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-tmp\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.307852    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovn-node-metrics-cert\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.309354 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.308046    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-tuned\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.309806 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.308731    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b0deac8-c535-4d85-b27e-968b22b55c14-agent-certs\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:45.362479 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.362443    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:21:45.362479 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.362478    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:21:45.362694 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.362493    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4jxsw for pod openshift-network-diagnostics/network-check-target-g9t4g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:45.362694 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.362601    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw podName:7ac23160-d159-4e38-9a63-691052ae8211 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:45.862582112 +0000 UTC m=+2.285425638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4jxsw" (UniqueName: "kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw") pod "network-check-target-g9t4g" (UID: "7ac23160-d159-4e38-9a63-691052ae8211") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:45.365678 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.365647    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlvlb\" (UniqueName: \"kubernetes.io/projected/dfd15a52-52ae-4718-a562-ba25778c39ea-kube-api-access-xlvlb\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.365797 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.365652    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz2b8\" (UniqueName: \"kubernetes.io/projected/8afabbf8-4122-42aa-95c6-75e6d41bf40d-kube-api-access-mz2b8\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.365856 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.365812    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmrdn\" (UniqueName: \"kubernetes.io/projected/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kube-api-access-qmrdn\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.365983 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.365953    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsjsb\" (UniqueName: \"kubernetes.io/projected/1f6b6450-cd73-4506-85d1-94ef030fe583-kube-api-access-zsjsb\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.366272 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.366253    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77hn5\" (UniqueName: \"kubernetes.io/projected/ce2518fb-0bf8-4628-9927-c46800cb2b3c-kube-api-access-77hn5\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.367008 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.366987    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9zhc\" (UniqueName: \"kubernetes.io/projected/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-kube-api-access-j9zhc\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.367316 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.367298    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s49km\" (UniqueName: \"kubernetes.io/projected/6d457444-8760-482f-a08d-7b20024db9fa-kube-api-access-s49km\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.367594 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.367568    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgckl\" (UniqueName: \"kubernetes.io/projected/ad0e377c-18f5-45ee-aabf-8b228f379db0-kube-api-access-wgckl\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:45.367808 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.367791    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn7q4\" (UniqueName: \"kubernetes.io/projected/8308da23-cdc6-4e50-91a2-9b158e17ca3b-kube-api-access-cn7q4\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.473173 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.473145    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:45.494533 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.494505    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:21:45.530107 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.530053    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc67bfe6e75d748c4abbd6f6df49a40.slice/crio-10ee490c492330638c8e951061b789a4aa3b7c2a65dfc99cf01239b14f512404 WatchSource:0}: Error finding container 10ee490c492330638c8e951061b789a4aa3b7c2a65dfc99cf01239b14f512404: Status 404 returned error can't find the container with id 10ee490c492330638c8e951061b789a4aa3b7c2a65dfc99cf01239b14f512404
Nov 03 22:21:45.530548 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.530534    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc6bc3da_6c5f_4bb6_9369_6967db1ad8a8.slice/crio-75258429ddbec7b2623724af3756989561043beb1332135c4f9f0d1eaa57ecc2 WatchSource:0}: Error finding container 75258429ddbec7b2623724af3756989561043beb1332135c4f9f0d1eaa57ecc2: Status 404 returned error can't find the container with id 75258429ddbec7b2623724af3756989561043beb1332135c4f9f0d1eaa57ecc2
Nov 03 22:21:45.535571 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.535556    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:21:45.549519 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.549478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:21:45.551641 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.551627    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rkgdk"
Nov 03 22:21:45.559666 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.559595    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f62230_4ed4_49ec_bc29_a69f0b46bd8f.slice/crio-843867e247b0c2a74e0f38cc87e1f4d8da0d1f0376d0d46ace77307cfcbcc5de WatchSource:0}: Error finding container 843867e247b0c2a74e0f38cc87e1f4d8da0d1f0376d0d46ace77307cfcbcc5de: Status 404 returned error can't find the container with id 843867e247b0c2a74e0f38cc87e1f4d8da0d1f0376d0d46ace77307cfcbcc5de
Nov 03 22:21:45.561731 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.561709    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd15a52_52ae_4718_a562_ba25778c39ea.slice/crio-af2f0fbaa530a742678f3ac999deb609fcafc4da133319a95deb6b099b2865ee WatchSource:0}: Error finding container af2f0fbaa530a742678f3ac999deb609fcafc4da133319a95deb6b099b2865ee: Status 404 returned error can't find the container with id af2f0fbaa530a742678f3ac999deb609fcafc4da133319a95deb6b099b2865ee
Nov 03 22:21:45.568985 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.568968    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:21:45.577785 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.577763    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afabbf8_4122_42aa_95c6_75e6d41bf40d.slice/crio-191b82c26c4c76cc0befe5731268667f0538656e71dac40c33228157ac33a6f9 WatchSource:0}: Error finding container 191b82c26c4c76cc0befe5731268667f0538656e71dac40c33228157ac33a6f9: Status 404 returned error can't find the container with id 191b82c26c4c76cc0befe5731268667f0538656e71dac40c33228157ac33a6f9
Nov 03 22:21:45.583066 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.583052    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:21:45.591935 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.591910    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2518fb_0bf8_4628_9927_c46800cb2b3c.slice/crio-5c02c6a23a3905c608facba35481a509275b5631d39f6356c6d73378f445ca63 WatchSource:0}: Error finding container 5c02c6a23a3905c608facba35481a509275b5631d39f6356c6d73378f445ca63: Status 404 returned error can't find the container with id 5c02c6a23a3905c608facba35481a509275b5631d39f6356c6d73378f445ca63
Nov 03 22:21:45.597147 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.597130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:21:45.602106 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.602040    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:21:45.605390 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.605364    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f6b6450_cd73_4506_85d1_94ef030fe583.slice/crio-ebe5c6dd23089ed9d8463b4e06475b271939884066a6d276d54070a5168eab99 WatchSource:0}: Error finding container ebe5c6dd23089ed9d8463b4e06475b271939884066a6d276d54070a5168eab99: Status 404 returned error can't find the container with id ebe5c6dd23089ed9d8463b4e06475b271939884066a6d276d54070a5168eab99
Nov 03 22:21:45.612797 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.612775    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d457444_8760_482f_a08d_7b20024db9fa.slice/crio-5492a31637d4407de7b75d2d3299f777b16d5325f1459be26d5b2cbad764207a WatchSource:0}: Error finding container 5492a31637d4407de7b75d2d3299f777b16d5325f1459be26d5b2cbad764207a: Status 404 returned error can't find the container with id 5492a31637d4407de7b75d2d3299f777b16d5325f1459be26d5b2cbad764207a
Nov 03 22:21:45.623395 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.623378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:21:45.639002 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.638979    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8308da23_cdc6_4e50_91a2_9b158e17ca3b.slice/crio-ce34cb7a203574d699b19f13cf706748ce59636979d65a0705b47e90cc44d813 WatchSource:0}: Error finding container ce34cb7a203574d699b19f13cf706748ce59636979d65a0705b47e90cc44d813: Status 404 returned error can't find the container with id ce34cb7a203574d699b19f13cf706748ce59636979d65a0705b47e90cc44d813
Nov 03 22:21:45.639598 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:45.639572    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46184df3bcae45c6ec28abb5e9a3b973.slice/crio-c944860f941df63d4d1b46a3a3e724fa8faa575e38245951d80f81bc02cf322b WatchSource:0}: Error finding container c944860f941df63d4d1b46a3a3e724fa8faa575e38245951d80f81bc02cf322b: Status 404 returned error can't find the container with id c944860f941df63d4d1b46a3a3e724fa8faa575e38245951d80f81bc02cf322b
Nov 03 22:21:45.808964 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.808874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:45.808964 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:45.808920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:45.809196 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.809008    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:45.809196 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.809010    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:45.809196 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.809066    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs podName:ad0e377c-18f5-45ee-aabf-8b228f379db0 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:46.809043406 +0000 UTC m=+3.231886915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs") pod "network-metrics-daemon-vr2lh" (UID: "ad0e377c-18f5-45ee-aabf-8b228f379db0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:45.809196 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:45.809107    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret podName:fd5c8298-1f37-4c24-bb52-1536b0e93028 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:46.809100296 +0000 UTC m=+3.231943802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret") pod "global-pull-secret-syncer-zhsp8" (UID: "fd5c8298-1f37-4c24-bb52-1536b0e93028") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:47.089968 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.089937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:47.090511 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:47.090082    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:21:47.090511 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:47.090097    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:21:47.090511 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:47.090106    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4jxsw for pod openshift-network-diagnostics/network-check-target-g9t4g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:47.090511 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:47.090148    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw podName:7ac23160-d159-4e38-9a63-691052ae8211 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:48.090136776 +0000 UTC m=+3.333388734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jxsw" (UniqueName: "kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw") pod "network-check-target-g9t4g" (UID: "7ac23160-d159-4e38-9a63-691052ae8211") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:47.216013 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.215961    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"46184df3bcae45c6ec28abb5e9a3b973","Type":"ContainerStarted","Data":"c944860f941df63d4d1b46a3a3e724fa8faa575e38245951d80f81bc02cf322b"}
Nov 03 22:21:47.217044 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.217019    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqjh7" event={"ID":"1f6b6450-cd73-4506-85d1-94ef030fe583","Type":"ContainerStarted","Data":"ebe5c6dd23089ed9d8463b4e06475b271939884066a6d276d54070a5168eab99"}
Nov 03 22:21:47.218078 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.218045    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-84hzv" event={"ID":"ce2518fb-0bf8-4628-9927-c46800cb2b3c","Type":"ContainerStarted","Data":"5c02c6a23a3905c608facba35481a509275b5631d39f6356c6d73378f445ca63"}
Nov 03 22:21:47.219095 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.219074    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2lfgw" event={"ID":"8afabbf8-4122-42aa-95c6-75e6d41bf40d","Type":"ContainerStarted","Data":"191b82c26c4c76cc0befe5731268667f0538656e71dac40c33228157ac33a6f9"}
Nov 03 22:21:47.220050 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.220033    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rkgdk" event={"ID":"dfd15a52-52ae-4718-a562-ba25778c39ea","Type":"ContainerStarted","Data":"af2f0fbaa530a742678f3ac999deb609fcafc4da133319a95deb6b099b2865ee"}
Nov 03 22:21:47.221028 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.221003    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerStarted","Data":"843867e247b0c2a74e0f38cc87e1f4d8da0d1f0376d0d46ace77307cfcbcc5de"}
Nov 03 22:21:47.222059 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.222040    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29" event={"ID":"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8","Type":"ContainerStarted","Data":"75258429ddbec7b2623724af3756989561043beb1332135c4f9f0d1eaa57ecc2"}
Nov 03 22:21:47.222984 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.222966    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"ce34cb7a203574d699b19f13cf706748ce59636979d65a0705b47e90cc44d813"}
Nov 03 22:21:47.223858 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.223841    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pvgnl" event={"ID":"6d457444-8760-482f-a08d-7b20024db9fa","Type":"ContainerStarted","Data":"5492a31637d4407de7b75d2d3299f777b16d5325f1459be26d5b2cbad764207a"}
Nov 03 22:21:47.224834 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.224816    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"9cc67bfe6e75d748c4abbd6f6df49a40","Type":"ContainerStarted","Data":"10ee490c492330638c8e951061b789a4aa3b7c2a65dfc99cf01239b14f512404"}
Nov 03 22:21:47.398934 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.398837    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 22:16:46 +0000 UTC" deadline="2028-08-21 06:57:59.532188564 +0000 UTC"
Nov 03 22:21:47.398934 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.398880    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15752h36m12.133310616s"
Nov 03 22:21:47.451163 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:21:47.451134    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0deac8_c535_4d85_b27e_968b22b55c14.slice/crio-4bd65c82c3fd805ef13820b6b0cd2447e2b385c4e9a40237e22fd8651bed0d66 WatchSource:0}: Error finding container 4bd65c82c3fd805ef13820b6b0cd2447e2b385c4e9a40237e22fd8651bed0d66: Status 404 returned error can't find the container with id 4bd65c82c3fd805ef13820b6b0cd2447e2b385c4e9a40237e22fd8651bed0d66
Nov 03 22:21:47.458193 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.458174    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:21:47.629601 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.629570    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:21:47.657302 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.657228    2576 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:21:47.999014 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.998898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:47.999014 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:47.998955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:47.999341 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:47.999108    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:47.999341 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:47.999165    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs podName:ad0e377c-18f5-45ee-aabf-8b228f379db0 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:49.999148479 +0000 UTC m=+5.242400438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs") pod "network-metrics-daemon-vr2lh" (UID: "ad0e377c-18f5-45ee-aabf-8b228f379db0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:47.999341 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:47.999227    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:47.999341 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:47.999259    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret podName:fd5c8298-1f37-4c24-bb52-1536b0e93028 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:49.999248352 +0000 UTC m=+5.242500313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret") pod "global-pull-secret-syncer-zhsp8" (UID: "fd5c8298-1f37-4c24-bb52-1536b0e93028") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:48.100052 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:48.100013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:48.100722 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:48.100165    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:21:48.100722 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:48.100184    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:21:48.100722 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:48.100196    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4jxsw for pod openshift-network-diagnostics/network-check-target-g9t4g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:48.100722 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:48.100254    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw podName:7ac23160-d159-4e38-9a63-691052ae8211 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:50.100237271 +0000 UTC m=+5.343489226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jxsw" (UniqueName: "kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw") pod "network-check-target-g9t4g" (UID: "7ac23160-d159-4e38-9a63-691052ae8211") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:48.212118 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:48.212072    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:48.212389 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:48.212197    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:21:48.212389 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:48.212293    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:48.212389 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:48.212375    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:21:48.212647 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:48.212430    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:48.212647 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:48.212499    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:21:48.254758 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:48.254660    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qj6mk" event={"ID":"0b0deac8-c535-4d85-b27e-968b22b55c14","Type":"ContainerStarted","Data":"4bd65c82c3fd805ef13820b6b0cd2447e2b385c4e9a40237e22fd8651bed0d66"}
Nov 03 22:21:48.399425 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:48.399356    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 22:16:46 +0000 UTC" deadline="2028-07-04 07:41:15.623685583 +0000 UTC"
Nov 03 22:21:48.399425 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:48.399420    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14601h19m27.224270117s"
Nov 03 22:21:50.015907 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:50.015876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:50.016399 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:50.015930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:50.016399 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.016078    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:50.016399 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.016131    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs podName:ad0e377c-18f5-45ee-aabf-8b228f379db0 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:54.01611817 +0000 UTC m=+9.259370138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs") pod "network-metrics-daemon-vr2lh" (UID: "ad0e377c-18f5-45ee-aabf-8b228f379db0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:50.016563 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.016474    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:50.016563 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.016523    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret podName:fd5c8298-1f37-4c24-bb52-1536b0e93028 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:54.01650845 +0000 UTC m=+9.259760422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret") pod "global-pull-secret-syncer-zhsp8" (UID: "fd5c8298-1f37-4c24-bb52-1536b0e93028") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:50.117296 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:50.116989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:50.117296 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.117166    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:21:50.117296 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.117185    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:21:50.117296 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.117199    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4jxsw for pod openshift-network-diagnostics/network-check-target-g9t4g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:50.117296 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.117255    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw podName:7ac23160-d159-4e38-9a63-691052ae8211 nodeName:}" failed. No retries permitted until 2026-11-03 22:21:54.117237816 +0000 UTC m=+9.360489786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jxsw" (UniqueName: "kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw") pod "network-check-target-g9t4g" (UID: "7ac23160-d159-4e38-9a63-691052ae8211") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:50.213340 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:50.212580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:50.213340 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.212724    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:21:50.213340 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:50.213096    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:50.213340 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.213192    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:21:50.213340 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:50.213237    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:50.213340 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:50.213306    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:21:51.277446 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:51.276787    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerStarted","Data":"5103f8e2da6d23a89e795d849335d0fdb5c54b67b1bdc733b6f399b50dea129e"}
Nov 03 22:21:52.213125 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.212406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:52.213125 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.212452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:52.213125 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:52.212538    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:21:52.213125 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:52.212969    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:21:52.213125 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.213026    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:52.213125 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:52.213095    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:21:52.283428 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.282724    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"9cc67bfe6e75d748c4abbd6f6df49a40","Type":"ContainerStarted","Data":"0acbab6299724cd0ebd6d25c163963ceb43d8cb3d6fc81886dd67bb3abf00c13"}
Nov 03 22:21:52.288337 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.287693    2576 generic.go:356] "Generic (PLEG): container finished" podID="46184df3bcae45c6ec28abb5e9a3b973" containerID="6f849f6e327ff30960c8f8b21fdeb4402f689a02fdb897456a86cb4064d04b34" exitCode=0
Nov 03 22:21:52.288337 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.287781    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"46184df3bcae45c6ec28abb5e9a3b973","Type":"ContainerDied","Data":"6f849f6e327ff30960c8f8b21fdeb4402f689a02fdb897456a86cb4064d04b34"}
Nov 03 22:21:52.293904 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.293287    2576 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="5103f8e2da6d23a89e795d849335d0fdb5c54b67b1bdc733b6f399b50dea129e" exitCode=0
Nov 03 22:21:52.293904 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.293330    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerDied","Data":"5103f8e2da6d23a89e795d849335d0fdb5c54b67b1bdc733b6f399b50dea129e"}
Nov 03 22:21:52.398969 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:52.398909    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal" podStartSLOduration=6.398888636 podStartE2EDuration="6.398888636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:52.397827479 +0000 UTC m=+7.641079497" watchObservedRunningTime="2026-11-03 22:21:52.398888636 +0000 UTC m=+7.642140622"
Nov 03 22:21:53.299334 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:53.298605    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qj6mk" event={"ID":"0b0deac8-c535-4d85-b27e-968b22b55c14","Type":"ContainerStarted","Data":"b6ae7d55c3ef4982cee0288d4896d51056150231035eb122e7bd5d25220bfc30"}
Nov 03 22:21:53.302529 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:53.302088    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"46184df3bcae45c6ec28abb5e9a3b973","Type":"ContainerStarted","Data":"db8f1c6ae76bedd85ccae941e5ee6c5683270ea92bb3c94bdd2ac2a071f33c54"}
Nov 03 22:21:53.307629 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:53.307459    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-84hzv" event={"ID":"ce2518fb-0bf8-4628-9927-c46800cb2b3c","Type":"ContainerStarted","Data":"0db83584f129b2e411ec0c2eb9d47493ccb974a0a033095957097bb10087b9b4"}
Nov 03 22:21:53.529549 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:53.529481    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qj6mk" podStartSLOduration=3.764660669 podStartE2EDuration="8.529462721s" totalImagesPullingTime="4.764802052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:53.414304668 +0000 UTC m=+8.657556645" watchObservedRunningTime="2026-11-03 22:21:53.529462721 +0000 UTC m=+8.772714699"
Nov 03 22:21:53.530537 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:53.530313    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal" podStartSLOduration=7.530302914 podStartE2EDuration="7.530302914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:53.528966634 +0000 UTC m=+8.772218612" watchObservedRunningTime="2026-11-03 22:21:53.530302914 +0000 UTC m=+8.773554892"
Nov 03 22:21:54.056537 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:54.056499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:54.056721 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:54.056560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:54.056802 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.056738    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:54.056853 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.056802    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs podName:ad0e377c-18f5-45ee-aabf-8b228f379db0 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:02.056784236 +0000 UTC m=+17.300036205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs") pod "network-metrics-daemon-vr2lh" (UID: "ad0e377c-18f5-45ee-aabf-8b228f379db0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:21:54.058113 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.057884    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:54.058113 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.057960    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret podName:fd5c8298-1f37-4c24-bb52-1536b0e93028 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:02.057942621 +0000 UTC m=+17.301194579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret") pod "global-pull-secret-syncer-zhsp8" (UID: "fd5c8298-1f37-4c24-bb52-1536b0e93028") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:21:54.157593 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:54.157222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:54.157593 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.157446    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:21:54.157593 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.157469    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:21:54.157593 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.157481    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4jxsw for pod openshift-network-diagnostics/network-check-target-g9t4g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:54.157593 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.157556    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw podName:7ac23160-d159-4e38-9a63-691052ae8211 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:02.157532518 +0000 UTC m=+17.400784489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jxsw" (UniqueName: "kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw") pod "network-check-target-g9t4g" (UID: "7ac23160-d159-4e38-9a63-691052ae8211") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:21:54.212174 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:54.212138    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:54.212427 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:54.212149    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:54.212427 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:54.212293    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:54.212427 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.212269    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:21:54.212427 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.212401    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:21:54.212838 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:54.212464    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:21:54.315158 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:54.315081    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29" event={"ID":"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8","Type":"ContainerStarted","Data":"5ec8c9999e01f1ddb6353c20700c62164674f65aa840b8fc141c7523c015dd32"}
Nov 03 22:21:55.244735 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:55.244563    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:55.245283 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:55.245108    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:55.321476 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:55.321439    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rkgdk" event={"ID":"dfd15a52-52ae-4718-a562-ba25778c39ea","Type":"ContainerStarted","Data":"31210206dcd4ebc18210132d8226a15daf488de9c3ef8a0a11b4644fe7459e7a"}
Nov 03 22:21:55.359933 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:55.359836    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-84hzv" podStartSLOduration=4.0796351 podStartE2EDuration="10.359822495s" totalImagesPullingTime="6.280187395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:53.643743279 +0000 UTC m=+8.886995258" watchObservedRunningTime="2026-11-03 22:21:55.359822495 +0000 UTC m=+10.603074533"
Nov 03 22:21:55.483220 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:55.483167    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-rkgdk" podStartSLOduration=2.124721287 podStartE2EDuration="10.48314865s" totalImagesPullingTime="8.358427363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:55.482933283 +0000 UTC m=+10.726185261" watchObservedRunningTime="2026-11-03 22:21:55.48314865 +0000 UTC m=+10.726400628"
Nov 03 22:21:55.515264 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:55.515238    2576 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 22:21:56.141257 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.141084    2576 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T22:21:55.515261954Z","UUID":"4935adb0-fdd9-45e5-a6eb-9a6a16e9db30","Handler":null,"Name":"","Endpoint":""}
Nov 03 22:21:56.143288 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.143016    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 22:21:56.143288 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.143054    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 22:21:56.212616 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.212578    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:56.212788 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.212695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:56.212788 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:56.212701    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:21:56.212856 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:56.212807    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:21:56.212896 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.212854    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:56.212955 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:56.212927    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:21:56.325004 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.324966    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29" event={"ID":"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8","Type":"ContainerStarted","Data":"b3dbbb1f7cf47f30d28658f53a39bb4c147bb6ba53973f108e7e47bbfea29675"}
Nov 03 22:21:56.327455 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.327426    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pvgnl" event={"ID":"6d457444-8760-482f-a08d-7b20024db9fa","Type":"ContainerStarted","Data":"317fe92ebdec124d7ce472fbdee0ecf21389f50b303499911bd01675cdb7b53c"}
Nov 03 22:21:56.333277 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:56.333249    2576 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 22:21:57.332913 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:57.332172    2576 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="abad503915913b63cfc8194a02b913c0239a930ad7338a27d4ff170200cc7dda" exitCode=0
Nov 03 22:21:57.332913 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:57.332490    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerDied","Data":"abad503915913b63cfc8194a02b913c0239a930ad7338a27d4ff170200cc7dda"}
Nov 03 22:21:57.336155 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:57.336116    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29" event={"ID":"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8","Type":"ContainerStarted","Data":"ffbfcbbae10d4dc28ef9fab85ac6b8a49f8091eaa20975a4b32178d3276d8630"}
Nov 03 22:21:57.339035 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:57.339007    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lqjh7" event={"ID":"1f6b6450-cd73-4506-85d1-94ef030fe583","Type":"ContainerStarted","Data":"2ef3eaf53845e971d4f5a3504a6f92658049c2f79cf9044ed7b46da79489506c"}
Nov 03 22:21:57.455205 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:57.455141    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-pvgnl" podStartSLOduration=3.530693679 podStartE2EDuration="12.45512226s" totalImagesPullingTime="8.924428581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:56.449659444 +0000 UTC m=+11.692911424" watchObservedRunningTime="2026-11-03 22:21:57.45512226 +0000 UTC m=+12.698374240"
Nov 03 22:21:57.573188 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:57.573131    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29" podStartSLOduration=2.651876446 podStartE2EDuration="12.573118204s" totalImagesPullingTime="9.921241758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:57.572857758 +0000 UTC m=+12.816109735" watchObservedRunningTime="2026-11-03 22:21:57.573118204 +0000 UTC m=+12.816370180"
Nov 03 22:21:57.688162 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:57.688040    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lqjh7" podStartSLOduration=3.9891751859999998 podStartE2EDuration="12.68802217s" totalImagesPullingTime="8.698846984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:57.687939751 +0000 UTC m=+12.931191730" watchObservedRunningTime="2026-11-03 22:21:57.68802217 +0000 UTC m=+12.931274148"
Nov 03 22:21:58.212576 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:58.212551    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:21:58.212707 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:58.212620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:21:58.212707 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:58.212646    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:21:58.212821 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:58.212774    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:21:58.212887 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:58.212862    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:21:58.212969 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:21:58.212946    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:21:58.344827 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:58.344793    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2lfgw" event={"ID":"8afabbf8-4122-42aa-95c6-75e6d41bf40d","Type":"ContainerStarted","Data":"f7f2d49ab84f4fa65c244707ce28fd6b1199ae00a4d5493b81d372a39df0bbab"}
Nov 03 22:21:58.923964 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:58.923870    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:58.924127 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:58.924050    2576 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 22:21:58.924767 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:58.924745    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:21:59.039561 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:59.039501    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2lfgw" podStartSLOduration=2.649000155 podStartE2EDuration="14.039482176s" totalImagesPullingTime="11.390482021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:21:58.465597442 +0000 UTC m=+13.708849420" watchObservedRunningTime="2026-11-03 22:21:59.039482176 +0000 UTC m=+14.282734157"
Nov 03 22:21:59.349485 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:59.349446    2576 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="68ce21292266f0898ead8b129c42f567f731ec3c7505307a2f954d0e77c45167" exitCode=0
Nov 03 22:21:59.350169 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:21:59.349493    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerDied","Data":"68ce21292266f0898ead8b129c42f567f731ec3c7505307a2f954d0e77c45167"}
Nov 03 22:22:00.212406 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:00.212363    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:00.212577 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:00.212411    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:00.212577 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:00.212512    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:00.212577 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:00.212548    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:00.212778 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:00.212593    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:00.212778 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:00.212686    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:01.357043 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:01.356817    2576 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="ce9f5a4aced8a190940e2c1852b53755cae978d86db479955e18dce91e710d51" exitCode=0
Nov 03 22:22:01.357571 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:01.356918    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerDied","Data":"ce9f5a4aced8a190940e2c1852b53755cae978d86db479955e18dce91e710d51"}
Nov 03 22:22:02.121977 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.121949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:02.122102 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.121987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:02.122102 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.122088    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:02.122102 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.122090    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:02.122198 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.122132    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs podName:ad0e377c-18f5-45ee-aabf-8b228f379db0 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:18.122120419 +0000 UTC m=+33.365372388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs") pod "network-metrics-daemon-vr2lh" (UID: "ad0e377c-18f5-45ee-aabf-8b228f379db0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:02.122198 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.122144    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret podName:fd5c8298-1f37-4c24-bb52-1536b0e93028 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:18.122138364 +0000 UTC m=+33.365390319 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret") pod "global-pull-secret-syncer-zhsp8" (UID: "fd5c8298-1f37-4c24-bb52-1536b0e93028") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:02.212720 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.212697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:02.212827 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.212693    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:02.212827 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.212807    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:02.212938 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.212835    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:02.212938 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.212915    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:02.213036 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.213007    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:02.223373 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.223347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:02.223536 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.223520    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:22:02.223599 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.223545    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:22:02.223599 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.223556    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4jxsw for pod openshift-network-diagnostics/network-check-target-g9t4g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:02.223706 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:02.223614    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw podName:7ac23160-d159-4e38-9a63-691052ae8211 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:18.22359556 +0000 UTC m=+33.466847517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jxsw" (UniqueName: "kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw") pod "network-check-target-g9t4g" (UID: "7ac23160-d159-4e38-9a63-691052ae8211") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:02.362970 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.362713    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:22:02.363433 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.363294    2576 generic.go:356] "Generic (PLEG): container finished" podID="8308da23-cdc6-4e50-91a2-9b158e17ca3b" containerID="b585e70638cb9743c8b29b916681f84f7105911e70bd6e25c8ccccc2adf8da5b" exitCode=1
Nov 03 22:22:02.363433 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.363354    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"e497168205914091853f74ff53c052fc651f28f4371cdb81c7c455522a80193d"}
Nov 03 22:22:02.363433 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.363417    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"5618f5aabb02749fc95471115f3fdd9165fd89f374508b37e4714e084fa0356e"}
Nov 03 22:22:02.363587 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.363447    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"c29b1bb3caf26c757e4c292e5520a501c5daecd1ad5c9c220a6591a2c1ded65b"}
Nov 03 22:22:02.363587 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.363483    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"89b7703eb7570d91637f95cc94fe171bcb50f6cb0a8c1ed22fd06b60789d4e9d"}
Nov 03 22:22:02.363587 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.363530    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerDied","Data":"b585e70638cb9743c8b29b916681f84f7105911e70bd6e25c8ccccc2adf8da5b"}
Nov 03 22:22:02.363587 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:02.363577    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"02492008b9a3ea3b62952eb7132ec540b549a946e916ca16e558a32fc72ce9fb"}
Nov 03 22:22:03.368284 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:03.368252    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerStarted","Data":"2fdf94163c19c6d3e2c79bc61dd6e5eb16f3b08eb555201cd213ebb0969c0375"}
Nov 03 22:22:04.211856 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:04.211815    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:04.212021 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:04.211869    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:04.212021 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:04.211965    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:04.212091 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:04.212039    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:04.212091 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:04.212055    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:04.212196 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:04.212100    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:04.373100 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:04.373065    2576 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="2fdf94163c19c6d3e2c79bc61dd6e5eb16f3b08eb555201cd213ebb0969c0375" exitCode=0
Nov 03 22:22:04.373574 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:04.373139    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerDied","Data":"2fdf94163c19c6d3e2c79bc61dd6e5eb16f3b08eb555201cd213ebb0969c0375"}
Nov 03 22:22:04.376571 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:04.376549    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:22:04.377007 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:04.376977    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"f186bd7b2e789d6a94ffdfc7e25306dc3a35e3f2adfb19f8b8b2b75e37b43d0f"}
Nov 03 22:22:05.382289 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:05.382252    2576 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="cd0a5df0dbb5290c6042ae0c497044453d297295ab97899a9f93ee710ac386a9" exitCode=0
Nov 03 22:22:05.382750 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:05.382296    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerDied","Data":"cd0a5df0dbb5290c6042ae0c497044453d297295ab97899a9f93ee710ac386a9"}
Nov 03 22:22:06.212508 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:06.212476    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:06.212695 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:06.212558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:06.212695 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:06.212651    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:06.212695 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:06.212683    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:06.212869 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:06.212771    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:06.212869 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:06.212857    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:06.388700 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:06.388638    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" event={"ID":"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f","Type":"ContainerStarted","Data":"237974c5adcdd3c13cdf642aafd458855c2585b0316d465cf960f8241dabec0a"}
Nov 03 22:22:06.556240 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:06.555964    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8ck4c" podStartSLOduration=9.490255897 podStartE2EDuration="21.55595183s" totalImagesPullingTime="12.065695933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:06.555730446 +0000 UTC m=+21.798982424" watchObservedRunningTime="2026-11-03 22:22:06.55595183 +0000 UTC m=+21.799203807"
Nov 03 22:22:06.765871 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:06.765841    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pvgnl_6d457444-8760-482f-a08d-7b20024db9fa/dns-node-resolver/0.log"
Nov 03 22:22:07.392339 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:07.392312    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-84hzv_ce2518fb-0bf8-4628-9927-c46800cb2b3c/node-ca/0.log"
Nov 03 22:22:07.394139 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:07.394124    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:22:07.394493 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:07.394473    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"eff4ca9fa4e856d47c3c7a4d2f4603dfcd3366dc4d2df60887a9302e060bfc89"}
Nov 03 22:22:07.395034 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:07.395018    2576 scope.go:122] "RemoveContainer" containerID="b585e70638cb9743c8b29b916681f84f7105911e70bd6e25c8ccccc2adf8da5b"
Nov 03 22:22:08.212527 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.212493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:08.212527 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.212520    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:08.212833 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.212493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:08.212833 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:08.212648    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:08.212833 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:08.212703    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:08.212833 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:08.212757    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:08.399974 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.399945    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:22:08.400348 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.400310    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" event={"ID":"8308da23-cdc6-4e50-91a2-9b158e17ca3b","Type":"ContainerStarted","Data":"5b92cdd053dbc22617ab59398a6e637013e99291739b8892663cc9cab829e9ac"}
Nov 03 22:22:08.400511 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.400496    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:08.400584 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.400553    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:08.400627 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.400602    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:08.415810 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.415785    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:08.415938 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.415889    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:08.606883 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:08.606781    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z" podStartSLOduration=8.649773575 podStartE2EDuration="23.606767721s" totalImagesPullingTime="14.956994146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:08.580497933 +0000 UTC m=+23.823749909" watchObservedRunningTime="2026-11-03 22:22:08.606767721 +0000 UTC m=+23.850019697"
Nov 03 22:22:10.212073 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:10.212034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:10.212530 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:10.212094    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:10.212530 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:10.212121    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:10.212530 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:10.212228    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:10.212530 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:10.212263    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:10.212530 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:10.212321    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:12.212853 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:12.212808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:12.213316 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:12.212947    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:12.213316 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:12.212961    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:12.213316 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:12.212973    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:12.213316 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:12.213025    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:12.213316 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:12.213078    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:14.212493 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:14.212465    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:14.212944 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:14.212500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:14.212944 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:14.212466    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:14.212944 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:14.212563    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:14.212944 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:14.212656    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:14.212944 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:14.212758    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:16.212646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:16.212599    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:16.212646 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:16.212599    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:16.213099 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:16.212622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:16.213099 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:16.212770    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:16.213099 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:16.212814    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:16.213099 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:16.212885    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:18.134813 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:18.134774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:18.134813 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:18.134816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:18.135246 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.134918    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:18.135246 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.134925    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:18.135246 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.134969    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs podName:ad0e377c-18f5-45ee-aabf-8b228f379db0 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:50.134956354 +0000 UTC m=+65.378208312 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs") pod "network-metrics-daemon-vr2lh" (UID: "ad0e377c-18f5-45ee-aabf-8b228f379db0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 22:22:18.135246 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.134988    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret podName:fd5c8298-1f37-4c24-bb52-1536b0e93028 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:50.134974962 +0000 UTC m=+65.378226917 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret") pod "global-pull-secret-syncer-zhsp8" (UID: "fd5c8298-1f37-4c24-bb52-1536b0e93028") : object "kube-system"/"original-pull-secret" not registered
Nov 03 22:22:18.212162 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:18.212120    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:18.212162 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:18.212154    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:18.212425 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:18.212120    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:18.212425 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.212244    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:18.212425 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.212296    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:18.212425 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.212355    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:18.235384 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:18.235353    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:18.235533 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.235516    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 22:22:18.235574 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.235541    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 22:22:18.235574 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.235553    2576 projected.go:196] Error preparing data for projected volume kube-api-access-4jxsw for pod openshift-network-diagnostics/network-check-target-g9t4g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:18.235633 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:18.235602    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw podName:7ac23160-d159-4e38-9a63-691052ae8211 nodeName:}" failed. No retries permitted until 2026-11-03 22:22:50.235589119 +0000 UTC m=+65.478841078 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jxsw" (UniqueName: "kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw") pod "network-check-target-g9t4g" (UID: "7ac23160-d159-4e38-9a63-691052ae8211") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 22:22:20.212320 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:20.212284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:20.212799 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:20.212398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:20.212799 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:20.212406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:20.212799 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:20.212403    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:20.212799 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:20.212502    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:20.212799 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:20.212582    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:22.212042 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:22.212006    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:22.212042 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:22.212021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:22.212494 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:22.212116    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:22.212494 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:22.212150    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:22.212494 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:22.212225    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:22.212494 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:22.212298    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:24.212184 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:24.212140    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:24.212561 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:24.212241    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:24.212561 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:24.212250    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:24.212561 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:24.212270    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:24.212561 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:24.212340    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:24.212561 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:24.212402    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:26.212819 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:26.212789    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:26.212819 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:26.212808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:26.213333 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:26.212800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:26.213333 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:26.212888    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:26.213333 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:26.212957    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:26.213333 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:26.213028    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:28.212192 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:28.212152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:28.212192 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:28.212180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:28.212706 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:28.212293    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:28.212706 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:28.212337    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:28.212706 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:28.212400    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:28.212706 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:28.212465    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:29.081994 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:29.081754    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zhsp8"]
Nov 03 22:22:29.082165 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:29.082073    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:29.082230 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:29.082188    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:29.082632 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:29.082598    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g9t4g"]
Nov 03 22:22:29.082773 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:29.082701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:29.082830 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:29.082784    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:29.084444 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:29.084416    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vr2lh"]
Nov 03 22:22:29.084557 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:29.084537    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:29.084684 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:29.084652    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:31.211937 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:31.211896    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:31.212556 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:31.211896    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:31.212556 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:31.212038    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9t4g" podUID="7ac23160-d159-4e38-9a63-691052ae8211"
Nov 03 22:22:31.212556 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:31.211902    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:31.212556 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:31.212094    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vr2lh" podUID="ad0e377c-18f5-45ee-aabf-8b228f379db0"
Nov 03 22:22:31.212556 ip-10-0-132-187 kubenswrapper[2576]: E1103 22:22:31.212197    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zhsp8" podUID="fd5c8298-1f37-4c24-bb52-1536b0e93028"
Nov 03 22:22:33.044884 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.044801    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeReady"
Nov 03 22:22:33.045226 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.044916    2576 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 03 22:22:33.160568 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.160531    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9fnc7"]
Nov 03 22:22:33.162256 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.162240    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.210452 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.210418    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lmz29"]
Nov 03 22:22:33.212454 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.212427    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.215491 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.215470    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z2ljt"]
Nov 03 22:22:33.215611 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.215602    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:33.215688 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.215652    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:33.215688 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.215648    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:33.217481 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.217463    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:33.239881 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.239860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-config-volume\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.240008 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.239900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-tmp-dir\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.240008 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.239997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-metrics-tls\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.240116 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.240055    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj9zk\" (UniqueName: \"kubernetes.io/projected/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-kube-api-access-fj9zk\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.242967 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.242945    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 22:22:33.243044 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.242981    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 22:22:33.243044 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.242998    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vc5c5\""
Nov 03 22:22:33.265226 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.265193    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 22:22:33.265365 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.265234    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-94lr2\""
Nov 03 22:22:33.265365 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.265262    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 22:22:33.265801 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.265787    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 22:22:33.266154 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.266141    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 22:22:33.272388 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.272363    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z2ljt"]
Nov 03 22:22:33.277104 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.277080    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9fnc7"]
Nov 03 22:22:33.293104 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.293068    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lmz29"]
Nov 03 22:22:33.341343 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341259    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkhzs\" (UniqueName: \"kubernetes.io/projected/485ddda6-f359-4513-9a89-239467e9f03a-kube-api-access-vkhzs\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:33.341343 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-config-volume\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.341343 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-tmp-dir\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.341549 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-metrics-tls\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.341549 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341522    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/395f30f1-f5f2-4164-90bd-02bfbaca7d31-crio-socket\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.341617 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.341617 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341586    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/395f30f1-f5f2-4164-90bd-02bfbaca7d31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.341784 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj9zk\" (UniqueName: \"kubernetes.io/projected/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-kube-api-access-fj9zk\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.341838 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341820    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/395f30f1-f5f2-4164-90bd-02bfbaca7d31-data-volume\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.341883 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341769    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-tmp-dir\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.341883 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z6th\" (UniqueName: \"kubernetes.io/projected/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-api-access-9z6th\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.341967 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485ddda6-f359-4513-9a89-239467e9f03a-cert\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:33.342012 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.341988    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-config-volume\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.345468 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.345442    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-metrics-tls\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.346320 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346302    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 22:22:33.346395 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346348    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 22:22:33.346447 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346406    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 22:22:33.346507 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346490    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-j25tw\""
Nov 03 22:22:33.346578 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346565    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ft6gs\""
Nov 03 22:22:33.346612 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346590    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 22:22:33.346662 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346650    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dnsfs\""
Nov 03 22:22:33.346725 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346662    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 22:22:33.346725 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346705    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-rrf2w\""
Nov 03 22:22:33.346790 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346743    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 22:22:33.346825 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.346802    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 22:22:33.402724 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.402693    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj9zk\" (UniqueName: \"kubernetes.io/projected/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-kube-api-access-fj9zk\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.443001 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.442960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/395f30f1-f5f2-4164-90bd-02bfbaca7d31-crio-socket\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.443001 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.442998    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.443211 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.443114    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/395f30f1-f5f2-4164-90bd-02bfbaca7d31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.443211 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.443181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/395f30f1-f5f2-4164-90bd-02bfbaca7d31-data-volume\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.443309 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.443209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z6th\" (UniqueName: \"kubernetes.io/projected/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-api-access-9z6th\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.443309 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.443206    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/395f30f1-f5f2-4164-90bd-02bfbaca7d31-crio-socket\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.443309 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.443289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485ddda6-f359-4513-9a89-239467e9f03a-cert\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:33.443503 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.443360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkhzs\" (UniqueName: \"kubernetes.io/projected/485ddda6-f359-4513-9a89-239467e9f03a-kube-api-access-vkhzs\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:33.443563 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.443536    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/395f30f1-f5f2-4164-90bd-02bfbaca7d31-data-volume\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.443621 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.443604    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.445780 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.445762    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/395f30f1-f5f2-4164-90bd-02bfbaca7d31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.445835 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.445817    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485ddda6-f359-4513-9a89-239467e9f03a-cert\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:33.473292 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.473256    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:33.504434 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.504398    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z6th\" (UniqueName: \"kubernetes.io/projected/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-api-access-9z6th\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.504728 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.504706    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkhzs\" (UniqueName: \"kubernetes.io/projected/485ddda6-f359-4513-9a89-239467e9f03a-kube-api-access-vkhzs\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:33.524656 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.524623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:33.546223 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.545922    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:33.852725 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.852593    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9fnc7"]
Nov 03 22:22:33.857823 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:33.857799    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe23ba2a_adba_4d25_b08c_cbcd0d0c0565.slice/crio-7f2dcccb375531f6bd24065f89da9ba70cdaf57ebbda802883fcbbec86e647e1 WatchSource:0}: Error finding container 7f2dcccb375531f6bd24065f89da9ba70cdaf57ebbda802883fcbbec86e647e1: Status 404 returned error can't find the container with id 7f2dcccb375531f6bd24065f89da9ba70cdaf57ebbda802883fcbbec86e647e1
Nov 03 22:22:33.902822 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.902789    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lmz29"]
Nov 03 22:22:33.906330 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:33.906302    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395f30f1_f5f2_4164_90bd_02bfbaca7d31.slice/crio-5c6cf1320dc0c5db26970952d43e0fd9a8f8893c2c3655dfcf489d266206bcd8 WatchSource:0}: Error finding container 5c6cf1320dc0c5db26970952d43e0fd9a8f8893c2c3655dfcf489d266206bcd8: Status 404 returned error can't find the container with id 5c6cf1320dc0c5db26970952d43e0fd9a8f8893c2c3655dfcf489d266206bcd8
Nov 03 22:22:33.952323 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:33.952293    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z2ljt"]
Nov 03 22:22:33.955336 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:33.955313    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485ddda6_f359_4513_9a89_239467e9f03a.slice/crio-c85e3f598d7a55956de744ce87deb19c74014854ac37926cfcab96feddeb9146 WatchSource:0}: Error finding container c85e3f598d7a55956de744ce87deb19c74014854ac37926cfcab96feddeb9146: Status 404 returned error can't find the container with id c85e3f598d7a55956de744ce87deb19c74014854ac37926cfcab96feddeb9146
Nov 03 22:22:34.457491 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:34.457456    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z2ljt" event={"ID":"485ddda6-f359-4513-9a89-239467e9f03a","Type":"ContainerStarted","Data":"c85e3f598d7a55956de744ce87deb19c74014854ac37926cfcab96feddeb9146"}
Nov 03 22:22:34.458856 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:34.458835    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lmz29" event={"ID":"395f30f1-f5f2-4164-90bd-02bfbaca7d31","Type":"ContainerStarted","Data":"6783c001eb7ff5384a704fc1e4f537fb4cce47eaf1d6f5e549874bfc6325e3b3"}
Nov 03 22:22:34.458956 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:34.458887    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lmz29" event={"ID":"395f30f1-f5f2-4164-90bd-02bfbaca7d31","Type":"ContainerStarted","Data":"5c6cf1320dc0c5db26970952d43e0fd9a8f8893c2c3655dfcf489d266206bcd8"}
Nov 03 22:22:34.459852 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:34.459831    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9fnc7" event={"ID":"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565","Type":"ContainerStarted","Data":"7f2dcccb375531f6bd24065f89da9ba70cdaf57ebbda802883fcbbec86e647e1"}
Nov 03 22:22:35.465958 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:35.465922    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lmz29" event={"ID":"395f30f1-f5f2-4164-90bd-02bfbaca7d31","Type":"ContainerStarted","Data":"fe59111dcf329ebbbd40b421da09dc73a0c97e5cbd7731d1e4f46f7c8b6c2146"}
Nov 03 22:22:36.470977 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.470936    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9fnc7" event={"ID":"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565","Type":"ContainerStarted","Data":"e4cc275f7fb22916a9d4d73ff9fdef4078a5406f476cab20b58322d3172b4e12"}
Nov 03 22:22:36.471480 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.471079    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9fnc7" event={"ID":"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565","Type":"ContainerStarted","Data":"ac2e1c83a7a725027cdacfb8c631f1f7e39518e0a40c034479aa280800b552f8"}
Nov 03 22:22:36.471480 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.471194    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:36.589148 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.589086    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-9fnc7" podStartSLOduration=1.44090182 podStartE2EDuration="3.589065182s" totalImagesPullingTime="2.148163362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:36.587912135 +0000 UTC m=+51.831164113" watchObservedRunningTime="2026-11-03 22:22:36.589065182 +0000 UTC m=+51.832317159"
Nov 03 22:22:36.857736 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.857638    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"]
Nov 03 22:22:36.859928 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.859902    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:36.913212 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.913083    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 03 22:22:36.913212 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.913147    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 22:22:36.913212 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.913174    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 22:22:36.913459 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.913306    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 03 22:22:36.913459 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.913321    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 22:22:36.913459 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.913425    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-frpm5\""
Nov 03 22:22:36.963949 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.963911    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"]
Nov 03 22:22:36.973234 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.973202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:36.973408 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.973274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhw5z\" (UniqueName: \"kubernetes.io/projected/01fe56b3-4ca9-416a-8480-e199577ac2d5-kube-api-access-nhw5z\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:36.973408 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.973358    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01fe56b3-4ca9-416a-8480-e199577ac2d5-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:36.973531 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:36.973433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.074421 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.074383    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.074582 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.074448    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhw5z\" (UniqueName: \"kubernetes.io/projected/01fe56b3-4ca9-416a-8480-e199577ac2d5-kube-api-access-nhw5z\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.074582 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.074493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01fe56b3-4ca9-416a-8480-e199577ac2d5-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.074582 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.074532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.075243 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.075192    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01fe56b3-4ca9-416a-8480-e199577ac2d5-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.077117 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.077093    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.077285 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.077267    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.104115 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.104090    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"]
Nov 03 22:22:37.106840 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.106823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.117408 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.117386    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8rxz5"]
Nov 03 22:22:37.119409 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.119394    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.141603 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.141571    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhw5z\" (UniqueName: \"kubernetes.io/projected/01fe56b3-4ca9-416a-8480-e199577ac2d5-kube-api-access-nhw5z\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.166330 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.166299    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"]
Nov 03 22:22:37.173946 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.173924    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:37.249094 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.249068    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q5zlz\""
Nov 03 22:22:37.249318 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.249190    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 03 22:22:37.249318 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.249257    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gnjcg\""
Nov 03 22:22:37.249318 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.249318    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 03 22:22:37.249524 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.249392    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 22:22:37.249524 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.249492    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 22:22:37.249786 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.249557    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 22:22:37.249786 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.249641    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 03 22:22:37.275836 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.275799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.275836 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.275838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-accelerators-collector-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.276090 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.275875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.276090 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.275892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.276090 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.275915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.276090 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.275949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-wtmp\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.276090 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.275974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-textfile\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.276090 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.276009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-metrics-client-ca\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.276090 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.276035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-root\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.276090 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.276063    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckdgq\" (UniqueName: \"kubernetes.io/projected/5db69683-41ca-4f88-a7ab-5a1174170725-kube-api-access-ckdgq\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.276469 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.276100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zsf4\" (UniqueName: \"kubernetes.io/projected/9e91ab02-2291-4937-b729-0422ee58256b-kube-api-access-8zsf4\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.276469 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.276129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-tls\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.276469 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.276160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9e91ab02-2291-4937-b729-0422ee58256b-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.276469 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.276192    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.276469 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.276215    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-sys\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.377593 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-tls\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.377593 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9e91ab02-2291-4937-b729-0422ee58256b-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.377593 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.377891 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-sys\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.377891 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.377891 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-accelerators-collector-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.377891 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377799    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-sys\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.377891 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.378130 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.378130 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.377994    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9e91ab02-2291-4937-b729-0422ee58256b-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.378130 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.378130 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-wtmp\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.378130 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-textfile\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.378130 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-metrics-client-ca\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.378581 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-root\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.378581 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckdgq\" (UniqueName: \"kubernetes.io/projected/5db69683-41ca-4f88-a7ab-5a1174170725-kube-api-access-ckdgq\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.378581 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378226    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zsf4\" (UniqueName: \"kubernetes.io/projected/9e91ab02-2291-4937-b729-0422ee58256b-kube-api-access-8zsf4\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.378581 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378355    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.378581 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378371    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-accelerators-collector-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.378581 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378451    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-root\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.378912 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.378850    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-wtmp\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.379336 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.379063    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-metrics-client-ca\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.379336 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.379139    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-textfile\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.379555 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.379530    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.380501 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.380485    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-tls\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.380501 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.380490    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.380881 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.380864    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.381377 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.381358    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.439525 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.439493    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckdgq\" (UniqueName: \"kubernetes.io/projected/5db69683-41ca-4f88-a7ab-5a1174170725-kube-api-access-ckdgq\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.439681 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.439604    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zsf4\" (UniqueName: \"kubernetes.io/projected/9e91ab02-2291-4937-b729-0422ee58256b-kube-api-access-8zsf4\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.475424 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.475369    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z2ljt" event={"ID":"485ddda6-f359-4513-9a89-239467e9f03a","Type":"ContainerStarted","Data":"753ba66224d042e3ebe20cdc3949a1d3ca1efd2db81b957440f6b096d21c40be"}
Nov 03 22:22:37.477455 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.477429    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lmz29" event={"ID":"395f30f1-f5f2-4164-90bd-02bfbaca7d31","Type":"ContainerStarted","Data":"d391029affb8ed8cbe0f4424c373a86a7382b8bab4a6c4686bcf167c9b731601"}
Nov 03 22:22:37.499191 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.499155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:37.516652 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.516620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:37.528049 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:37.528015    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db69683_41ca_4f88_a7ab_5a1174170725.slice/crio-17f89a2e3a273ff2a67baf7c40109cf5229c488a5d7f8e295931f2d2f7bd8154 WatchSource:0}: Error finding container 17f89a2e3a273ff2a67baf7c40109cf5229c488a5d7f8e295931f2d2f7bd8154: Status 404 returned error can't find the container with id 17f89a2e3a273ff2a67baf7c40109cf5229c488a5d7f8e295931f2d2f7bd8154
Nov 03 22:22:37.595286 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.595263    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"]
Nov 03 22:22:37.597270 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:37.597247    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01fe56b3_4ca9_416a_8480_e199577ac2d5.slice/crio-80af86e253ef2ce44fd81537e34b5fd7fcb82251254ba410abad1e947a7c62c0 WatchSource:0}: Error finding container 80af86e253ef2ce44fd81537e34b5fd7fcb82251254ba410abad1e947a7c62c0: Status 404 returned error can't find the container with id 80af86e253ef2ce44fd81537e34b5fd7fcb82251254ba410abad1e947a7c62c0
Nov 03 22:22:37.600302 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.599362    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z2ljt" podStartSLOduration=1.461614003 podStartE2EDuration="4.599346365s" totalImagesPullingTime="3.137732362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:37.599065549 +0000 UTC m=+52.842317526" watchObservedRunningTime="2026-11-03 22:22:37.599346365 +0000 UTC m=+52.842598343"
Nov 03 22:22:37.719121 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.719070    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lmz29" podStartSLOduration=1.72318971 podStartE2EDuration="4.719054906s" totalImagesPullingTime="2.995865196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:37.718717329 +0000 UTC m=+52.961969307" watchObservedRunningTime="2026-11-03 22:22:37.719054906 +0000 UTC m=+52.962306879"
Nov 03 22:22:37.883308 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:37.883273    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"]
Nov 03 22:22:37.886389 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:37.886358    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e91ab02_2291_4937_b729_0422ee58256b.slice/crio-f03fcb31d36e5fc094fa5beb818c5b0a4266aaf72071b2deef43e60e7cc4a91c WatchSource:0}: Error finding container f03fcb31d36e5fc094fa5beb818c5b0a4266aaf72071b2deef43e60e7cc4a91c: Status 404 returned error can't find the container with id f03fcb31d36e5fc094fa5beb818c5b0a4266aaf72071b2deef43e60e7cc4a91c
Nov 03 22:22:38.481021 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:38.480978    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8rxz5" event={"ID":"5db69683-41ca-4f88-a7ab-5a1174170725","Type":"ContainerStarted","Data":"17f89a2e3a273ff2a67baf7c40109cf5229c488a5d7f8e295931f2d2f7bd8154"}
Nov 03 22:22:38.482869 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:38.482839    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7" event={"ID":"01fe56b3-4ca9-416a-8480-e199577ac2d5","Type":"ContainerStarted","Data":"bf8f5048fd53c2ca9cbd948041022d84245d3233176bb6c5d5b3706b21d4093f"}
Nov 03 22:22:38.483001 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:38.482903    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7" event={"ID":"01fe56b3-4ca9-416a-8480-e199577ac2d5","Type":"ContainerStarted","Data":"2e0370ab45a328c6cb9c274bd10501497300737b27fb60d7d6a22e648893fe3c"}
Nov 03 22:22:38.483001 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:38.482931    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7" event={"ID":"01fe56b3-4ca9-416a-8480-e199577ac2d5","Type":"ContainerStarted","Data":"80af86e253ef2ce44fd81537e34b5fd7fcb82251254ba410abad1e947a7c62c0"}
Nov 03 22:22:38.484035 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:38.483987    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb" event={"ID":"9e91ab02-2291-4937-b729-0422ee58256b","Type":"ContainerStarted","Data":"f03fcb31d36e5fc094fa5beb818c5b0a4266aaf72071b2deef43e60e7cc4a91c"}
Nov 03 22:22:39.240111 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.240072    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:22:39.242279 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.242254    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.260626 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.260586    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:22:39.309346 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.309318    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 03 22:22:39.310331 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.310312    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-84pw6\""
Nov 03 22:22:39.310407 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.310383    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 03 22:22:39.310463 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.310439    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 03 22:22:39.310528 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.310513    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 03 22:22:39.310582 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.310559    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 03 22:22:39.310682 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.310631    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 03 22:22:39.310734 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.310695    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 03 22:22:39.310791 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.310757    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 03 22:22:39.396235 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396413 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396413 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f768m\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396413 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396342    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396413 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396388    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396604 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396420    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396604 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396604 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396604 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396604 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396538    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.396604 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396567    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 03 22:22:39.396604 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.396579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.397297 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.397070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.397297 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.397134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.490786 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.490713    2576 generic.go:356] "Generic (PLEG): container finished" podID="5db69683-41ca-4f88-a7ab-5a1174170725" containerID="319e36fc27aa84d5ec18bb03473e16e24a664fb9b7ca0947d3ec23c716eb1301" exitCode=0
Nov 03 22:22:39.490786 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.490771    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8rxz5" event={"ID":"5db69683-41ca-4f88-a7ab-5a1174170725","Type":"ContainerDied","Data":"319e36fc27aa84d5ec18bb03473e16e24a664fb9b7ca0947d3ec23c716eb1301"}
Nov 03 22:22:39.498503 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.498617 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.498617 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.498617 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498582    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.498777 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.498777 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.498777 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.498777 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.498777 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.499000 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f768m\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.499000 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.499000 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.499000 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.498894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.500381 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.500350    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.500639 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.500616    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.502200 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.502180    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.502282 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.502215    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.502501 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.502432    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.503135 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.503108    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.503972 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.503947    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.505177 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.505154    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.505295 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.505245    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.506161 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.506140    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.506355 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.506335    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.508365 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.508344    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.560522 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.560492    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f768m\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.562267 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.562244    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:39.584731 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.584486    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"]
Nov 03 22:22:39.586953 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.586932    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.646050 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.645240    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 03 22:22:39.646050 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.645319    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 03 22:22:39.646050 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.645245    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 03 22:22:39.646050 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.645541    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-14p5jl2nv912m\""
Nov 03 22:22:39.646050 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.645826    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rfslm\""
Nov 03 22:22:39.646272 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.646117    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 03 22:22:39.646737 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.646509    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 03 22:22:39.664317 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.664280    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"]
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92z8d\" (UniqueName: \"kubernetes.io/projected/3ead9dc1-5785-432c-a28d-6dd002c4dc27-kube-api-access-92z8d\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-grpc-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ead9dc1-5785-432c-a28d-6dd002c4dc27-metrics-client-ca\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ead9dc1-5785-432c-a28d-6dd002c4dc27-tmp\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700364    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700399    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.700544 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.700469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801520 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801620 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801620 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92z8d\" (UniqueName: \"kubernetes.io/projected/3ead9dc1-5785-432c-a28d-6dd002c4dc27-kube-api-access-92z8d\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801783 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-grpc-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801783 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ead9dc1-5785-432c-a28d-6dd002c4dc27-metrics-client-ca\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801783 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801783 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801964 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ead9dc1-5785-432c-a28d-6dd002c4dc27-tmp\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.801964 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.801830    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.802778 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.802747    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ead9dc1-5785-432c-a28d-6dd002c4dc27-tmp\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.803490 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.803465    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ead9dc1-5785-432c-a28d-6dd002c4dc27-metrics-client-ca\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.804271 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.804251    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.804997 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.804968    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.805089 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.805068    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.805470 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.805452    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.806244 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.806228    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-grpc-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.807569 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.807552    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.864499 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.864466    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92z8d\" (UniqueName: \"kubernetes.io/projected/3ead9dc1-5785-432c-a28d-6dd002c4dc27-kube-api-access-92z8d\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:39.909798 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:39.909769    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:40.007782 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.007700    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:22:40.010473 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:40.010442    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea0760a_ec14_4dce_b11b_d746d72de9ee.slice/crio-a26ae6a44a25759ab38d917b596be6f63ade0ef65561db450fb7468662e5717e WatchSource:0}: Error finding container a26ae6a44a25759ab38d917b596be6f63ade0ef65561db450fb7468662e5717e: Status 404 returned error can't find the container with id a26ae6a44a25759ab38d917b596be6f63ade0ef65561db450fb7468662e5717e
Nov 03 22:22:40.287445 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.287420    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"]
Nov 03 22:22:40.417368 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.417338    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:40.495912 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.495877    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8rxz5" event={"ID":"5db69683-41ca-4f88-a7ab-5a1174170725","Type":"ContainerStarted","Data":"10c0f55985a8281c487698f14441c85dc56af3fdba0bf187df3a108dc078498e"}
Nov 03 22:22:40.496330 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.495941    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8rxz5" event={"ID":"5db69683-41ca-4f88-a7ab-5a1174170725","Type":"ContainerStarted","Data":"2fb9d0442769977d841f8aa632915c248e9667dcee59e6d6dc1b03e86ede7bd3"}
Nov 03 22:22:40.498118 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.498086    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7" event={"ID":"01fe56b3-4ca9-416a-8480-e199577ac2d5","Type":"ContainerStarted","Data":"ff02393b01c2c5bb736a11f5125b54e80426721d323c91c45649d0a9214b5c98"}
Nov 03 22:22:40.499355 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.499328    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerStarted","Data":"a26ae6a44a25759ab38d917b596be6f63ade0ef65561db450fb7468662e5717e"}
Nov 03 22:22:40.500450 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.500429    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" event={"ID":"3ead9dc1-5785-432c-a28d-6dd002c4dc27","Type":"ContainerStarted","Data":"79988fe6f2d4e25bf75a438d7305fcc6444424973573ce3f0cb1475e2a0e9c83"}
Nov 03 22:22:40.502514 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.502496    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb" event={"ID":"9e91ab02-2291-4937-b729-0422ee58256b","Type":"ContainerStarted","Data":"435a22bc11af21ad6fe9f5525ff35908b9b23e938dca2a133cdd830be54b5bec"}
Nov 03 22:22:40.502620 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.502531    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb" event={"ID":"9e91ab02-2291-4937-b729-0422ee58256b","Type":"ContainerStarted","Data":"aaad62bb61ffd6eb1da071edaa7bd0c27ca4716001cf789f27b5e4482974cf91"}
Nov 03 22:22:40.502620 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.502548    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb" event={"ID":"9e91ab02-2291-4937-b729-0422ee58256b","Type":"ContainerStarted","Data":"7cb4e54c2bca43b7746f3a7aa224aed7a4d8a6fb9c3aa58fe48c4451f8d79a9f"}
Nov 03 22:22:40.665846 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.665734    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8rxz5" podStartSLOduration=1.8392317409999999 podStartE2EDuration="3.665720271s" totalImagesPullingTime="1.741415095s" totalInitContainerRuntime="85.073435ms" isStatefulPod=true podCreationTimestamp="2026-11-03 22:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:40.664990347 +0000 UTC m=+55.908242318" watchObservedRunningTime="2026-11-03 22:22:40.665720271 +0000 UTC m=+55.908972247"
Nov 03 22:22:40.784214 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.784122    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7" podStartSLOduration=3.086296687 podStartE2EDuration="4.784104504s" totalImagesPullingTime="1.697807817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:40.783747037 +0000 UTC m=+56.026999018" watchObservedRunningTime="2026-11-03 22:22:40.784104504 +0000 UTC m=+56.027356485"
Nov 03 22:22:40.902062 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:40.902007    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb" podStartSLOduration=2.107740211 podStartE2EDuration="3.901990941s" totalImagesPullingTime="1.79425073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:40.901408674 +0000 UTC m=+56.144660654" watchObservedRunningTime="2026-11-03 22:22:40.901990941 +0000 UTC m=+56.145242920"
Nov 03 22:22:41.508376 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.508331    2576 generic.go:356] "Generic (PLEG): container finished" podID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerID="0f11b7165af4bc6c0c544820b2a21aa30ecc72f833193c1e810261b33cc3a926" exitCode=0
Nov 03 22:22:41.508853 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.508432    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerDied","Data":"0f11b7165af4bc6c0c544820b2a21aa30ecc72f833193c1e810261b33cc3a926"}
Nov 03 22:22:41.924731 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.924695    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6ddc555bd7-sljjr"]
Nov 03 22:22:41.926904 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.926881    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:41.985417 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.985382    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 03 22:22:41.985723 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.985704    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 03 22:22:41.991060 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.991035    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-z56st\""
Nov 03 22:22:41.991440 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.991422    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fagdf02i9kd0c\""
Nov 03 22:22:41.994817 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.994796    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 03 22:22:41.994951 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:41.994878    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 03 22:22:42.009396 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.009366    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6ddc555bd7-sljjr"]
Nov 03 22:22:42.023548 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.023511    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-client-certs\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.023704 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.023559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-client-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.023704 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.023655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-tls\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.023797 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.023739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-metrics-server-audit-profiles\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.023797 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.023786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.023852 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.023805    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mvqp\" (UniqueName: \"kubernetes.io/projected/4196c457-6d8d-4a01-b299-653d0b4e4445-kube-api-access-9mvqp\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.023852 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.023830    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4196c457-6d8d-4a01-b299-653d0b4e4445-audit-log\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.124999 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.124965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-metrics-server-audit-profiles\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.125161 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.125020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.125161 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.125039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mvqp\" (UniqueName: \"kubernetes.io/projected/4196c457-6d8d-4a01-b299-653d0b4e4445-kube-api-access-9mvqp\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.125161 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.125062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4196c457-6d8d-4a01-b299-653d0b4e4445-audit-log\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.125161 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.125088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-client-certs\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.125161 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.125113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-client-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.125161 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.125152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-tls\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.126415 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.126365    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.126690 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.126649    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4196c457-6d8d-4a01-b299-653d0b4e4445-audit-log\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.126984 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.126844    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-metrics-server-audit-profiles\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.128088 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.128051    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-tls\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.130138 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.130118    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-client-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.132055 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.132037    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-client-certs\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.188971 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.188897    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mvqp\" (UniqueName: \"kubernetes.io/projected/4196c457-6d8d-4a01-b299-653d0b4e4445-kube-api-access-9mvqp\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.239475 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.239234    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:42.662578 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:42.662545    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6ddc555bd7-sljjr"]
Nov 03 22:22:42.672442 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:42.669817    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4196c457_6d8d_4a01_b299_653d0b4e4445.slice/crio-56109620006a9d0e0aea12cf87f4414b5d3370ebbc6ca80ff5d6eb72f17de557 WatchSource:0}: Error finding container 56109620006a9d0e0aea12cf87f4414b5d3370ebbc6ca80ff5d6eb72f17de557: Status 404 returned error can't find the container with id 56109620006a9d0e0aea12cf87f4414b5d3370ebbc6ca80ff5d6eb72f17de557
Nov 03 22:22:43.521900 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:43.521869    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerStarted","Data":"7225ca7443734aaffc19b8897de44628fd1036915289bfbb09c67cf5a72fe50f"}
Nov 03 22:22:43.522017 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:43.521999    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerStarted","Data":"4aeb105f7e82e22064df94e06995a158c27c9ab5aa6f5fa009e03842578e61e4"}
Nov 03 22:22:43.522094 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:43.522044    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerStarted","Data":"97282f71b3cba780ab4aa230ad6714a6e2b9d9aeef9ac63697becf1c34bfeac1"}
Nov 03 22:22:43.525394 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:43.525369    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" event={"ID":"3ead9dc1-5785-432c-a28d-6dd002c4dc27","Type":"ContainerStarted","Data":"8d1be27faca6d39b9789ec206abb08479ad239ae0d2ec2b85fa5ce26041df8a0"}
Nov 03 22:22:43.525511 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:43.525440    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" event={"ID":"3ead9dc1-5785-432c-a28d-6dd002c4dc27","Type":"ContainerStarted","Data":"b4bbb6e8aff3e5ffc1a1126ff027747f7a2094bcdd6eed2ee6b704e0a39a45dc"}
Nov 03 22:22:43.525511 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:43.525484    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" event={"ID":"3ead9dc1-5785-432c-a28d-6dd002c4dc27","Type":"ContainerStarted","Data":"e50bfec2f93c68f358626288d6dc06c6e615ded19d2b5dd9175edda8580f7bee"}
Nov 03 22:22:43.526787 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:43.526728    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr" event={"ID":"4196c457-6d8d-4a01-b299-653d0b4e4445","Type":"ContainerStarted","Data":"56109620006a9d0e0aea12cf87f4414b5d3370ebbc6ca80ff5d6eb72f17de557"}
Nov 03 22:22:44.535600 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.535558    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" event={"ID":"3ead9dc1-5785-432c-a28d-6dd002c4dc27","Type":"ContainerStarted","Data":"3860f0d3046281f7d387c1092217d9e57ccbb0345eb1390583a5fa679ab44cb2"}
Nov 03 22:22:44.536102 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.535637    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" event={"ID":"3ead9dc1-5785-432c-a28d-6dd002c4dc27","Type":"ContainerStarted","Data":"e09fb8b8a2f4a640e8eeb0995b3e1a56de501313dc3abcc1ad727d87ec1c5d3a"}
Nov 03 22:22:44.536102 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.535694    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" event={"ID":"3ead9dc1-5785-432c-a28d-6dd002c4dc27","Type":"ContainerStarted","Data":"1d2a3155eb42485b54cafb716e535f8aa7d49987da1711a75e0016462f38d17c"}
Nov 03 22:22:44.536102 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.535745    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:44.537151 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.537125    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr" event={"ID":"4196c457-6d8d-4a01-b299-653d0b4e4445","Type":"ContainerStarted","Data":"35b34c6e28e559d45af7205a0ff9078329fd827b1355d6d6b095ee21f047c0c1"}
Nov 03 22:22:44.540183 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.540161    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerStarted","Data":"ff97d24c7d876bebbbb7fa344ad849586f1b7aafb1b6b2c0c2d39b5ea5c29f24"}
Nov 03 22:22:44.540275 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.540227    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerStarted","Data":"6f2aff40ecbe743b52300cb09538554c1aef616b9c9c34407c2874075f911d3b"}
Nov 03 22:22:44.661190 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.661090    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" podStartSLOduration=2.002217388 podStartE2EDuration="5.661074592s" totalImagesPullingTime="3.658857204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:44.659750279 +0000 UTC m=+59.903002273" watchObservedRunningTime="2026-11-03 22:22:44.661074592 +0000 UTC m=+59.904326569"
Nov 03 22:22:44.777348 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:44.777285    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr" podStartSLOduration=2.044007058 podStartE2EDuration="3.777268894s" totalImagesPullingTime="1.733261836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:44.776999048 +0000 UTC m=+60.020251025" watchObservedRunningTime="2026-11-03 22:22:44.777268894 +0000 UTC m=+60.020520872"
Nov 03 22:22:45.546558 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:45.546520    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerStarted","Data":"e07c32aa30f619b5c5bb7d57bdb0d82ff3898f6750c0b3d74a66c409f21dedca"}
Nov 03 22:22:45.725506 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:45.725452    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.524108779 podStartE2EDuration="6.725434662s" totalImagesPullingTime="4.180457988s" totalInitContainerRuntime="20.867895ms" isStatefulPod=false podCreationTimestamp="2026-11-03 22:22:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:45.724542834 +0000 UTC m=+60.967794811" watchObservedRunningTime="2026-11-03 22:22:45.725434662 +0000 UTC m=+60.968686639"
Nov 03 22:22:46.479475 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:46.479446    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:50.196239 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.196203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:50.196613 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.196282    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:50.247958 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.247925    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 22:22:50.259599 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.259573    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:50.296861 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.296826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:50.297995 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.297977    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 22:22:50.309296 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.309269    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:50.348483 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.348455    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 22:22:50.381318 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.381293    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dnsfs\""
Nov 03 22:22:50.390632 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.390614    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:50.392679 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.392653    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ft6gs\""
Nov 03 22:22:50.402081 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.402064    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:50.408574 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.408549    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 22:22:50.420385 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.420359    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:50.553748 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.553722    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:50.728108 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.728073    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-j25tw\""
Nov 03 22:22:50.735403 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.735372    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:50.839477 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.839452    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zhsp8"]
Nov 03 22:22:50.842193 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:50.842168    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vr2lh"]
Nov 03 22:22:50.842290 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:50.842254    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd5c8298_1f37_4c24_bb52_1536b0e93028.slice/crio-3b22f90c3432e8bfe0f733b5962a010f658c099106a9042cd41fcebce903bb53 WatchSource:0}: Error finding container 3b22f90c3432e8bfe0f733b5962a010f658c099106a9042cd41fcebce903bb53: Status 404 returned error can't find the container with id 3b22f90c3432e8bfe0f733b5962a010f658c099106a9042cd41fcebce903bb53
Nov 03 22:22:50.844479 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:50.844455    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0e377c_18f5_45ee_aabf_8b228f379db0.slice/crio-39d8576dbab5cc662bb4c00dd3439a07053bc31b80ef03dcfbec8048d0a8bbd1 WatchSource:0}: Error finding container 39d8576dbab5cc662bb4c00dd3439a07053bc31b80ef03dcfbec8048d0a8bbd1: Status 404 returned error can't find the container with id 39d8576dbab5cc662bb4c00dd3439a07053bc31b80ef03dcfbec8048d0a8bbd1
Nov 03 22:22:51.103605 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:51.103513    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g9t4g"]
Nov 03 22:22:51.106231 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:51.106196    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac23160_d159_4e38_9a63_691052ae8211.slice/crio-c6fe8e5169818a54cf1ac46ad312e7b1d5aa6e84767dd98e5227f1be59e84c5a WatchSource:0}: Error finding container c6fe8e5169818a54cf1ac46ad312e7b1d5aa6e84767dd98e5227f1be59e84c5a: Status 404 returned error can't find the container with id c6fe8e5169818a54cf1ac46ad312e7b1d5aa6e84767dd98e5227f1be59e84c5a
Nov 03 22:22:51.575301 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:51.575267    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vr2lh" event={"ID":"ad0e377c-18f5-45ee-aabf-8b228f379db0","Type":"ContainerStarted","Data":"39d8576dbab5cc662bb4c00dd3439a07053bc31b80ef03dcfbec8048d0a8bbd1"}
Nov 03 22:22:51.576423 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:51.576393    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g9t4g" event={"ID":"7ac23160-d159-4e38-9a63-691052ae8211","Type":"ContainerStarted","Data":"c6fe8e5169818a54cf1ac46ad312e7b1d5aa6e84767dd98e5227f1be59e84c5a"}
Nov 03 22:22:51.577378 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:51.577359    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zhsp8" event={"ID":"fd5c8298-1f37-4c24-bb52-1536b0e93028","Type":"ContainerStarted","Data":"3b22f90c3432e8bfe0f733b5962a010f658c099106a9042cd41fcebce903bb53"}
Nov 03 22:22:52.586702 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:52.586655    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vr2lh" event={"ID":"ad0e377c-18f5-45ee-aabf-8b228f379db0","Type":"ContainerStarted","Data":"c61548e81c2268caef6e088dfaecfa582772f072c1aac359ad474f01306241c9"}
Nov 03 22:22:53.503154 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.503123    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-59c56bd6bd-82w6k"]
Nov 03 22:22:53.506368 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.506345    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.594493 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.594169    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vr2lh" event={"ID":"ad0e377c-18f5-45ee-aabf-8b228f379db0","Type":"ContainerStarted","Data":"0ee8437e070165f9075303990a2b13f9570b17168ed42fd8d22496e2891d705c"}
Nov 03 22:22:53.609797 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.609748    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 03 22:22:53.610049 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.610009    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mtgdn\""
Nov 03 22:22:53.610049 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.610043    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 03 22:22:53.610290 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.610267    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 03 22:22:53.610290 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.610271    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 03 22:22:53.610445 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.610287    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 22:22:53.610445 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.610316    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 03 22:22:53.610445 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.610346    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 22:22:53.628694 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.628638    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.628862 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.628701    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.628862 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.628760    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.628947 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.628870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.628947 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.628939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.629001 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.628969    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.629170 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.629149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh6vv\" (UniqueName: \"kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.663272 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.663237    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c56bd6bd-82w6k"]
Nov 03 22:22:53.730069 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.730027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh6vv\" (UniqueName: \"kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.730226 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.730094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.730226 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.730133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.730226 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.730158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.730226 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.730193    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.730226 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.730220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.730415 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.730244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.731041 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.731020    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.731883 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.731861    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.732733 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.732526    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.733564 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.733534    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.734836 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.734812    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.735002 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.734983    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.796459 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.796387    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh6vv\" (UniqueName: \"kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:53.821244 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:53.821207    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:54.259939 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:54.259866    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vr2lh" podStartSLOduration=67.706157832 podStartE2EDuration="1m9.259843843s" totalImagesPullingTime="1.553686011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:53.779108717 +0000 UTC m=+69.022360698" watchObservedRunningTime="2026-11-03 22:22:54.259843843 +0000 UTC m=+69.503095822"
Nov 03 22:22:54.260286 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:54.260264    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c56bd6bd-82w6k"]
Nov 03 22:22:54.264730 ip-10-0-132-187 kubenswrapper[2576]: W1103 22:22:54.264633    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod789cc7c0_4a41_4117_a79f_a2c9ec5b7ec3.slice/crio-3b00ce6e5e5dc82bd9429956df4ea8a97fadea907ef3efce57e2974c925cd8f1 WatchSource:0}: Error finding container 3b00ce6e5e5dc82bd9429956df4ea8a97fadea907ef3efce57e2974c925cd8f1: Status 404 returned error can't find the container with id 3b00ce6e5e5dc82bd9429956df4ea8a97fadea907ef3efce57e2974c925cd8f1
Nov 03 22:22:54.600462 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:54.600372    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c56bd6bd-82w6k" event={"ID":"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3","Type":"ContainerStarted","Data":"3b00ce6e5e5dc82bd9429956df4ea8a97fadea907ef3efce57e2974c925cd8f1"}
Nov 03 22:22:55.606879 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:55.606831    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g9t4g" event={"ID":"7ac23160-d159-4e38-9a63-691052ae8211","Type":"ContainerStarted","Data":"29530c12e2994f4ca8783c51615e64a9a3afb7ae00a9d89f21af51ec9ab5bb90"}
Nov 03 22:22:55.607392 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:55.606996    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:55.722889 ip-10-0-132-187 kubenswrapper[2576]: I1103 22:22:55.722840    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g9t4g" podStartSLOduration=67.121836216 podStartE2EDuration="1m10.722819623s" totalImagesPullingTime="3.600983407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:22:55.721581231 +0000 UTC m=+70.964833209" watchObservedRunningTime="2026-11-03 22:22:55.722819623 +0000 UTC m=+70.966071601"
Nov 03 22:22:56.507395 ip-10-0-132-187 systemd[1]: Stopping Kubernetes Kubelet...
Nov 03 22:22:56.516197 ip-10-0-132-187 systemd[1]: kubelet.service: Deactivated successfully.
Nov 03 22:22:56.516425 ip-10-0-132-187 systemd[1]: Stopped Kubernetes Kubelet.
Nov 03 22:22:56.516483 ip-10-0-132-187 systemd[1]: kubelet.service: Consumed 4.274s CPU time.
Nov 03 22:22:56.524345 ip-10-0-132-187 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 22:22:56.593868 ip-10-0-132-187 kubenswrapper[5647]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:22:56.593868 ip-10-0-132-187 kubenswrapper[5647]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 22:22:56.593868 ip-10-0-132-187 kubenswrapper[5647]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:22:56.593868 ip-10-0-132-187 kubenswrapper[5647]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 22:22:56.596573 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596482    5647 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:56.596573 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596574    5647 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:56.596573 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596580    5647 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596587    5647 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596590    5647 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596594    5647 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596598    5647 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596603    5647 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596607    5647 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596611    5647 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596615    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596617    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596620    5647 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596623    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596625    5647 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596628    5647 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596630    5647 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596633    5647 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596636    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596638    5647 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596640    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596642    5647 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:56.596834 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596645    5647 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596648    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596653    5647 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596657    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596663    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596684    5647 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596688    5647 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596692    5647 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596695    5647 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596699    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596703    5647 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596706    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596709    5647 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596712    5647 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596717    5647 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596719    5647 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596722    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596725    5647 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596727    5647 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596730    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:56.597654 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596732    5647 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596736    5647 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596740    5647 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596742    5647 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596745    5647 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596748    5647 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596751    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596753    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596756    5647 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596758    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596761    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596763    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596765    5647 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596768    5647 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596770    5647 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596773    5647 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596775    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596779    5647 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596783    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:56.598504 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596788    5647 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596791    5647 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596794    5647 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596798    5647 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596802    5647 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596806    5647 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596811    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596817    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596822    5647 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596827    5647 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596831    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596836    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596841    5647 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596845    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596848    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596852    5647 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596857    5647 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596861    5647 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596866    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:56.599311 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596869    5647 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596871    5647 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596874    5647 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596877    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596881    5647 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596884    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596887    5647 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596890    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596893    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596895    5647 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596898    5647 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596901    5647 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596903    5647 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596906    5647 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596908    5647 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596911    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596914    5647 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596917    5647 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596919    5647 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596922    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:56.600112 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596926    5647 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596928    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596931    5647 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596936    5647 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596941    5647 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.596944    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597826    5647 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597834    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597837    5647 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597841    5647 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597845    5647 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597848    5647 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597850    5647 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597852    5647 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597855    5647 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597857    5647 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597860    5647 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597863    5647 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597865    5647 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:56.600997 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597867    5647 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597870    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597872    5647 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597875    5647 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597877    5647 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597880    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597883    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597888    5647 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597892    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597896    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597903    5647 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597908    5647 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597911    5647 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597917    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597920    5647 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597923    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597925    5647 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597928    5647 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597931    5647 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:56.601794 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597933    5647 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597936    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597939    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597941    5647 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597944    5647 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597947    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597950    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597953    5647 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597955    5647 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597958    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597960    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597962    5647 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597965    5647 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597967    5647 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597970    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597975    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597979    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597985    5647 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597988    5647 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597990    5647 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:56.602619 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597994    5647 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597996    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.597999    5647 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598001    5647 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598004    5647 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598007    5647 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598012    5647 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598017    5647 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598021    5647 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598024    5647 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598027    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598029    5647 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598032    5647 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598035    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598037    5647 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598040    5647 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598044    5647 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598049    5647 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598055    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:56.603449 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598062    5647 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598067    5647 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598070    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598073    5647 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598075    5647 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598078    5647 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598080    5647 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598083    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598086    5647 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598088    5647 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598091    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598094    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598097    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598099    5647 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598102    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598104    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598107    5647 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598109    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598112    5647 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:56.604249 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598118    5647 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598120    5647 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598123    5647 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598125    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598128    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598130    5647 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598133    5647 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598135    5647 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598138    5647 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598140    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598143    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598145    5647 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598147    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598154    5647 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598157    5647 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598160    5647 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598738    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598746    5647 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598750    5647 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598753    5647 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:56.605068 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598756    5647 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598759    5647 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598761    5647 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598764    5647 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598767    5647 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598769    5647 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598772    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598775    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598777    5647 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598780    5647 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598782    5647 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598785    5647 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598790    5647 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598792    5647 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598795    5647 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598797    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598800    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598804    5647 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598807    5647 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:56.605905 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598810    5647 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598813    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598815    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598818    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598820    5647 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598823    5647 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598826    5647 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598829    5647 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598831    5647 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598834    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598836    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598839    5647 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598841    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598845    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598848    5647 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598850    5647 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598853    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598856    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598860    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:56.606750 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598864    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598869    5647 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598874    5647 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598878    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598880    5647 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598883    5647 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598887    5647 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598890    5647 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598892    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598896    5647 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598898    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598901    5647 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598903    5647 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598906    5647 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598908    5647 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598911    5647 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598914    5647 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598916    5647 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598918    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598921    5647 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:56.607543 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598924    5647 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598927    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598929    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598931    5647 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598934    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598937    5647 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598940    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598943    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598945    5647 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598947    5647 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598950    5647 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598953    5647 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598955    5647 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598957    5647 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598960    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598962    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598965    5647 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598967    5647 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598971    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:56.608366 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598974    5647 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598976    5647 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598979    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598981    5647 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598984    5647 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598987    5647 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598990    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598995    5647 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.598999    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599002    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599004    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599007    5647 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599009    5647 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599012    5647 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599015    5647 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599018    5647 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599020    5647 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599023    5647 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599025    5647 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:56.609169 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599029    5647 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599031    5647 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599034    5647 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599037    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599039    5647 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599043    5647 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599150    5647 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599161    5647 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599170    5647 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599175    5647 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599179    5647 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599183    5647 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599189    5647 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599196    5647 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599201    5647 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599206    5647 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599211    5647 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599214    5647 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599217    5647 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599220    5647 flags.go:64] FLAG: --cgroup-root=""
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599223    5647 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599226    5647 flags.go:64] FLAG: --client-ca-file=""
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599229    5647 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 22:22:56.609975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599231    5647 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599235    5647 flags.go:64] FLAG: --cluster-domain=""
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599238    5647 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599240    5647 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599243    5647 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599248    5647 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599252    5647 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599255    5647 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599258    5647 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599262    5647 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599265    5647 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599268    5647 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599271    5647 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599277    5647 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599282    5647 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599289    5647 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599292    5647 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599295    5647 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599298    5647 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599301    5647 flags.go:64] FLAG: --enable-server="true"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599304    5647 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599309    5647 flags.go:64] FLAG: --event-burst="100"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599313    5647 flags.go:64] FLAG: --event-qps="50"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599316    5647 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 22:22:56.610992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599319    5647 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599322    5647 flags.go:64] FLAG: --eviction-hard=""
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599326    5647 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599329    5647 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599332    5647 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599335    5647 flags.go:64] FLAG: --eviction-soft=""
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599338    5647 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599341    5647 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599344    5647 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599347    5647 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599351    5647 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599355    5647 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599360    5647 flags.go:64] FLAG: --feature-gates=""
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599366    5647 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599369    5647 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599373    5647 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599376    5647 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599379    5647 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599383    5647 flags.go:64] FLAG: --help="false"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599385    5647 flags.go:64] FLAG: --hostname-override="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599388    5647 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599391    5647 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599394    5647 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 22:22:56.612040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599397    5647 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599401    5647 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599404    5647 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599407    5647 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599409    5647 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599412    5647 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599416    5647 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599419    5647 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599421    5647 flags.go:64] FLAG: --kube-reserved=""
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599424    5647 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599448    5647 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599452    5647 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599455    5647 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599458    5647 flags.go:64] FLAG: --lock-file=""
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599461    5647 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599464    5647 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599467    5647 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599472    5647 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599475    5647 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599478    5647 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599481    5647 flags.go:64] FLAG: --logging-format="text"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599483    5647 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599487    5647 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599489    5647 flags.go:64] FLAG: --manifest-url=""
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599493    5647 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 22:22:56.613032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599497    5647 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599500    5647 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599504    5647 flags.go:64] FLAG: --max-pods="110"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599507    5647 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599511    5647 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599516    5647 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599521    5647 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599526    5647 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599529    5647 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599531    5647 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599538    5647 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599541    5647 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599544    5647 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599547    5647 flags.go:64] FLAG: --pod-cidr=""
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599550    5647 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599553    5647 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599556    5647 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599559    5647 flags.go:64] FLAG: --port="10250"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599561    5647 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599564    5647 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-06f62ae6ad0a33845"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599567    5647 flags.go:64] FLAG: --qos-reserved=""
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599570    5647 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599573    5647 flags.go:64] FLAG: --register-node="true"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599576    5647 flags.go:64] FLAG: --register-with-taints=""
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599579    5647 flags.go:64] FLAG: --registry-burst="10"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599582    5647 flags.go:64] FLAG: --registry-qps="5"
Nov 03 22:22:56.614076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599585    5647 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599587    5647 flags.go:64] FLAG: --reserved-memory=""
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599591    5647 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599594    5647 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599597    5647 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599600    5647 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599603    5647 flags.go:64] FLAG: --runonce="false"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599606    5647 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599609    5647 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599612    5647 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599615    5647 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599618    5647 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599621    5647 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599625    5647 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599628    5647 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599630    5647 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599633    5647 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599636    5647 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599639    5647 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599643    5647 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599648    5647 flags.go:64] FLAG: --system-cgroups=""
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599653    5647 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599660    5647 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599678    5647 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599682    5647 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 22:22:56.615170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599685    5647 flags.go:64] FLAG: --tls-min-version=""
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599688    5647 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599691    5647 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599693    5647 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599697    5647 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599699    5647 flags.go:64] FLAG: --v="2"
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599704    5647 flags.go:64] FLAG: --version="false"
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599708    5647 flags.go:64] FLAG: --vmodule=""
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599712    5647 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.599715    5647 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599863    5647 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599868    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599871    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599874    5647 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599877    5647 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599881    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599885    5647 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599890    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599894    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599898    5647 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599902    5647 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599906    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:56.616328 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599909    5647 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599911    5647 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599914    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599916    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599919    5647 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599921    5647 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599924    5647 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599927    5647 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599929    5647 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599931    5647 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599934    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599936    5647 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599939    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599941    5647 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599944    5647 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599946    5647 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599951    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599953    5647 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599956    5647 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:56.617268 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599959    5647 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599961    5647 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599964    5647 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599966    5647 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599968    5647 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599971    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599974    5647 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599976    5647 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599979    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599981    5647 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599984    5647 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599986    5647 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599989    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599991    5647 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599994    5647 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599996    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.599998    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600002    5647 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600004    5647 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:56.618079 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600007    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600009    5647 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600012    5647 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600016    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600020    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600024    5647 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600028    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600032    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600037    5647 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600042    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600048    5647 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600053    5647 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600057    5647 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600062    5647 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600066    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600071    5647 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600076    5647 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600080    5647 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600084    5647 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600088    5647 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:56.618933 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600093    5647 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600097    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600103    5647 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600110    5647 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600115    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600119    5647 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600124    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600131    5647 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600137    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600142    5647 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600146    5647 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600149    5647 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600152    5647 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600154    5647 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600157    5647 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600160    5647 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600162    5647 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600167    5647 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600171    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:56.619843 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600175    5647 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600180    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600184    5647 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600190    5647 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600195    5647 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600199    5647 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600204    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600208    5647 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600212    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600217    5647 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600220    5647 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600222    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600225    5647 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600227    5647 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600231    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600233    5647 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:56.620735 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600236    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.600267    5647 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600303    5647 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600308    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600310    5647 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600313    5647 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600315    5647 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600318    5647 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600322    5647 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600324    5647 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600327    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600331    5647 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600335    5647 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600338    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600341    5647 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600343    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:56.621397 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600347    5647 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600351    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600354    5647 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600358    5647 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600361    5647 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600364    5647 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600366    5647 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600369    5647 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600371    5647 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600374    5647 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600376    5647 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600379    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600381    5647 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600384    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600386    5647 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600389    5647 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600392    5647 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600395    5647 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600399    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600404    5647 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:56.622086 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600408    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600412    5647 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600414    5647 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600417    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600423    5647 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600427    5647 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600432    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600436    5647 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600440    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600442    5647 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600445    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600447    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600450    5647 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600453    5647 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600455    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600460    5647 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600463    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600465    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600468    5647 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600470    5647 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:56.622925 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600473    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600477    5647 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600481    5647 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600485    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600490    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600495    5647 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600497    5647 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600500    5647 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600504    5647 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600506    5647 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600509    5647 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600511    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600514    5647 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600517    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600519    5647 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600521    5647 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600526    5647 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600528    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600531    5647 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:56.623782 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600533    5647 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600536    5647 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600539    5647 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600541    5647 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600544    5647 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600546    5647 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600548    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600551    5647 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600554    5647 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600557    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600570    5647 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600574    5647 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600577    5647 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600580    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600585    5647 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600589    5647 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600594    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600598    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600603    5647 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600607    5647 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:56.624577 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600612    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600616    5647 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600621    5647 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600626    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600630    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600634    5647 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600639    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600643    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600648    5647 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600654    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600656    5647 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600659    5647 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.600662    5647 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.607724    5647 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.607741    5647 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.607760    5647 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.607766    5647 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607917    5647 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607923    5647 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:56.625401 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607926    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607929    5647 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607932    5647 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607935    5647 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607937    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607940    5647 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607943    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607946    5647 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607949    5647 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607951    5647 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607957    5647 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607959    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607963    5647 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607965    5647 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607968    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607971    5647 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607974    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607976    5647 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607979    5647 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:56.626214 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607981    5647 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607984    5647 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607987    5647 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607992    5647 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607995    5647 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.607998    5647 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608000    5647 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608005    5647 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608010    5647 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608015    5647 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608019    5647 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608023    5647 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608027    5647 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608031    5647 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608037    5647 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608042    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608048    5647 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608051    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608054    5647 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608056    5647 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:56.627008 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608059    5647 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608063    5647 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608067    5647 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608070    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608073    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608076    5647 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608079    5647 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608082    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608087    5647 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608089    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608092    5647 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608095    5647 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608097    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608100    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608103    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608106    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608109    5647 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608111    5647 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608114    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:56.627853 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608116    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608122    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608125    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608128    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608131    5647 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608133    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608136    5647 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608138    5647 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608141    5647 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608143    5647 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608146    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608149    5647 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608151    5647 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608154    5647 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608159    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608161    5647 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608163    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608166    5647 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608168    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:56.628638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608171    5647 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608173    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608176    5647 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608179    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608181    5647 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608183    5647 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608186    5647 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608189    5647 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608194    5647 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608196    5647 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608200    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608204    5647 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608208    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608212    5647 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608216    5647 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608220    5647 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608224    5647 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608228    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608232    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:56.629441 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608236    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608239    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608244    5647 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608247    5647 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608249    5647 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608252    5647 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608254    5647 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608257    5647 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.608292    5647 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608600    5647 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608608    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608613    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608617    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608621    5647 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608627    5647 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:56.630298 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608633    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608637    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608642    5647 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608645    5647 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608650    5647 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608653    5647 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608656    5647 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608658    5647 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608660    5647 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608663    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608683    5647 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608687    5647 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608691    5647 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608695    5647 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608710    5647 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608844    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608850    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608854    5647 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608857    5647 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:56.630930 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608862    5647 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608866    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608871    5647 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608875    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608879    5647 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608885    5647 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608891    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608896    5647 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608902    5647 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608906    5647 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608910    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608914    5647 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608918    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608922    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608926    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608930    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608934    5647 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608938    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608942    5647 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:56.631725 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608946    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608951    5647 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608954    5647 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608958    5647 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608965    5647 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608969    5647 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608973    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608981    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608987    5647 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608992    5647 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.608996    5647 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609000    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609005    5647 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609009    5647 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609013    5647 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609017    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609021    5647 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609026    5647 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609030    5647 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:56.632424 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609034    5647 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609039    5647 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609043    5647 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609047    5647 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609051    5647 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609055    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609060    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609064    5647 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609069    5647 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609073    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609079    5647 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609083    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609087    5647 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609091    5647 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609095    5647 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609100    5647 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609105    5647 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609111    5647 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609116    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609119    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:56.633223 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609122    5647 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609126    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609128    5647 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609131    5647 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609134    5647 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609136    5647 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609139    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609141    5647 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609144    5647 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609146    5647 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609149    5647 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609151    5647 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609154    5647 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609156    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609158    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609161    5647 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609163    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609166    5647 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609168    5647 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609171    5647 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:56.634061 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609175    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609187    5647 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609192    5647 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.609228    5647 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609266    5647 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609270    5647 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609273    5647 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609276    5647 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609279    5647 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609282    5647 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609285    5647 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609287    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609290    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609295    5647 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609297    5647 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609300    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 22:22:56.634926 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609302    5647 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609305    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609307    5647 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609310    5647 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609312    5647 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609315    5647 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609317    5647 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609334    5647 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609338    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609342    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609344    5647 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609347    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609350    5647 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609352    5647 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609355    5647 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609357    5647 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609360    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609362    5647 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609370    5647 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609373    5647 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 22:22:56.635405 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609375    5647 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609377    5647 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609380    5647 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609382    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609385    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609389    5647 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609394    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609398    5647 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609401    5647 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609404    5647 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609407    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609409    5647 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609411    5647 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609414    5647 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609416    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609419    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609421    5647 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609424    5647 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609428    5647 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 22:22:56.636075 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609432    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609435    5647 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609438    5647 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609440    5647 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609443    5647 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609445    5647 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609448    5647 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609450    5647 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609452    5647 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609455    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609457    5647 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609460    5647 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609467    5647 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609470    5647 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609473    5647 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609475    5647 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609478    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609480    5647 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609482    5647 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 22:22:56.636836 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609485    5647 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609487    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609490    5647 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609493    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609496    5647 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609501    5647 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609504    5647 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609507    5647 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609510    5647 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609513    5647 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609516    5647 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609518    5647 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609521    5647 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609523    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609526    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609528    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609531    5647 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609533    5647 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609536    5647 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 22:22:56.637423 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609538    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609541    5647 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609543    5647 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609546    5647 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609548    5647 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609550    5647 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609553    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609562    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609565    5647 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609568    5647 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609572    5647 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609577    5647 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609582    5647 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609587    5647 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609591    5647 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609595    5647 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:22:56.609602    5647 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.609714    5647 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.610652    5647 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 03 22:22:56.638036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.610726    5647 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.610950    5647 server.go:1014] "Starting client certificate rotation"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.611095    5647 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.611150    5647 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 22:16:45 +0000 UTC" deadline="2028-08-05 16:55:41.358260036 +0000 UTC"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.611196    5647 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15378h32m44.747067329s"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.611551    5647 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.612898    5647 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.614398    5647 log.go:25] "Validated CRI v1 runtime API"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.616526    5647 log.go:25] "Validated CRI v1 image API"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.617355    5647 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 22:22:56.638903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.619802    5647 fs.go:135] Filesystem UUIDs: map[761f7cf9-0f4b-4f70-b121-df7a570b9e3e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 98c4c6fe-bd7d-470e-a46f-2345f99e7201:/dev/nvme0n1p4]
Nov 03 22:22:56.639378 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.619821    5647 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/10ee490c492330638c8e951061b789a4aa3b7c2a65dfc99cf01239b14f512404/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/10ee490c492330638c8e951061b789a4aa3b7c2a65dfc99cf01239b14f512404/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/17f89a2e3a273ff2a67baf7c40109cf5229c488a5d7f8e295931f2d2f7bd8154/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/17f89a2e3a273ff2a67baf7c40109cf5229c488a5d7f8e295931f2d2f7bd8154/userdata/shm major:0 minor:263 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/39d8576dbab5cc662bb4c00dd3439a07053bc31b80ef03dcfbec8048d0a8bbd1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/39d8576dbab5cc662bb4c00dd3439a07053bc31b80ef03dcfbec8048d0a8bbd1/userdata/shm major:0 minor:418 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3b00ce6e5e5dc82bd9429956df4ea8a97fadea907ef3efce57e2974c925cd8f1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3b00ce6e5e5dc82bd9429956df4ea8a97fadea907ef3efce57e2974c925cd8f1/userdata/shm major:0 minor:443 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3b22f90c3432e8bfe0f733b5962a010f658c099106a9042cd41fcebce903bb53/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3b22f90c3432e8bfe0f733b5962a010f658c099106a9042cd41fcebce903bb53/userdata/shm major:0 minor:416 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4bd65c82c3fd805ef13820b6b0cd2447e2b385c4e9a40237e22fd8651bed0d66/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4bd65c82c3fd805ef13820b6b0cd2447e2b385c4e9a40237e22fd8651bed0d66/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5492a31637d4407de7b75d2d3299f777b16d5325f1459be26d5b2cbad764207a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5492a31637d4407de7b75d2d3299f777b16d5325f1459be26d5b2cbad764207a/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/56109620006a9d0e0aea12cf87f4414b5d3370ebbc6ca80ff5d6eb72f17de557/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/56109620006a9d0e0aea12cf87f4414b5d3370ebbc6ca80ff5d6eb72f17de557/userdata/shm major:0 minor:342 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5c02c6a23a3905c608facba35481a509275b5631d39f6356c6d73378f445ca63/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5c02c6a23a3905c608facba35481a509275b5631d39f6356c6d73378f445ca63/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5c6cf1320dc0c5db26970952d43e0fd9a8f8893c2c3655dfcf489d266206bcd8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5c6cf1320dc0c5db26970952d43e0fd9a8f8893c2c3655dfcf489d266206bcd8/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/75258429ddbec7b2623724af3756989561043beb1332135c4f9f0d1eaa57ecc2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/75258429ddbec7b2623724af3756989561043beb1332135c4f9f0d1eaa57ecc2/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/79988fe6f2d4e25bf75a438d7305fcc6444424973573ce3f0cb1475e2a0e9c83/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/79988fe6f2d4e25bf75a438d7305fcc6444424973573ce3f0cb1475e2a0e9c83/userdata/shm major:0 minor:332 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7f2dcccb375531f6bd24065f89da9ba70cdaf57ebbda802883fcbbec86e647e1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7f2dcccb375531f6bd24065f89da9ba70cdaf57ebbda802883fcbbec86e647e1/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/80af86e253ef2ce44fd81537e34b5fd7fcb82251254ba410abad1e947a7c62c0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/80af86e253ef2ce44fd81537e34b5fd7fcb82251254ba410abad1e947a7c62c0/userdata/shm major:0 minor:249 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/843867e247b0c2a74e0f38cc87e1f4d8da0d1f0376d0d46ace77307cfcbcc5de/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/843867e247b0c2a74e0f38cc87e1f4d8da0d1f0376d0d46ace77307cfcbcc5de/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a26ae6a44a25759ab38d917b596be6f63ade0ef65561db450fb7468662e5717e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a26ae6a44a25759ab38d917b596be6f63ade0ef65561db450fb7468662e5717e/userdata/shm major:0 minor:303 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/af2f0fbaa530a742678f3ac999deb609fcafc4da133319a95deb6b099b2865ee/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/af2f0fbaa530a742678f3ac999deb609fcafc4da133319a95deb6b099b2865ee/userdata/shm major:0 minor:71 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c6fe8e5169818a54cf1ac46ad312e7b1d5aa6e84767dd98e5227f1be59e84c5a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c6fe8e5169818a54cf1ac46ad312e7b1d5aa6e84767dd98e5227f1be59e84c5a/userdata/shm major:0 minor:421 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c85e3f598d7a55956de744ce87deb19c74014854ac37926cfcab96feddeb9146/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c85e3f598d7a55956de744ce87deb19c74014854ac37926cfcab96feddeb9146/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c944860f941df63d4d1b46a3a3e724fa8faa575e38245951d80f81bc02cf322b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c944860f941df63d4d1b46a3a3e724fa8faa575e38245951d80f81bc02cf322b/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce34cb7a203574d699b19f13cf706748ce59636979d65a0705b47e90cc44d813/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce34cb7a203574d699b19f13cf706748ce59636979d65a0705b47e90cc44d813/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ebe5c6dd23089ed9d8463b4e06475b271939884066a6d276d54070a5168eab99/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ebe5c6dd23089ed9d8463b4e06475b271939884066a6d276d54070a5168eab99/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f03fcb31d36e5fc094fa5beb818c5b0a4266aaf72071b2deef43e60e7cc4a91c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f03fcb31d36e5fc094fa5beb818c5b0a4266aaf72071b2deef43e60e7cc4a91c/userdata/shm major:0 minor:261 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~projected/kube-api-access-nhw5z:{mountpoint:/var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~projected/kube-api-access-nhw5z major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:242 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b0deac8-c535-4d85-b27e-968b22b55c14/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/0b0deac8-c535-4d85-b27e-968b22b55c14/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f6b6450-cd73-4506-85d1-94ef030fe583/volumes/kubernetes.io~projected/kube-api-access-zsjsb:{mountpoint:/var/lib/kubelet/pods/1f6b6450-cd73-4506-85d1-94ef030fe583/volumes/kubernetes.io~projected/kube-api-access-zsjsb major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/395f30f1-f5f2-4164-90bd-02bfbaca7d31/volumes/kubernetes.io~projected/kube-api-access-9z6th:{mountpoint:/var/lib/kubelet/pods/395f30f1-f5f2-4164-90bd-02bfbaca7d31/volumes/kubernetes.io~projected/kube-api-access-9z6th major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/395f30f1-f5f2-4164-90bd-02bfbaca7d31/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/395f30f1-f5f2-4164-90bd-02bfbaca7d31/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~projected/kube-api-access-92z8d:{mountpoint:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~projected/kube-api-access-92z8d major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~projected/kube-api-access-9mvqp:{mountpoint:/var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~projected/kube-api-access-9mvqp major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:338 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/485ddda6-f359-4513-9a89-239467e9f03a/volumes/kubernetes.io~projected/kube-api-access-vkhzs:{mountpoint:/var/lib/kubelet/pods/485ddda6-f359-4513-9a89-239467e9f03a/volumes/kubernetes.io~projected/kube-api-access-vkhzs major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/485ddda6-f359-4513-9a89-239467e9f03a/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/485ddda6-f359-4513-9a89-239467e9f03a/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~projected/kube-api-access-ckdgq:{mountpoint:/var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~projected/kube-api-access-ckdgq major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:256 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d457444-8760-482f-a08d-7b20024db9fa/volumes/kubernetes.io~projected/kube-api-access-s49km:{mountpoint:/var/lib/kubelet/pods/6d457444-8760-482f-a08d-7b20024db9fa/volumes/kubernetes.io~projected/kube-api-access-s49km major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~projected/kube-api-access-zh6vv:{mountpoint:/var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~projected/kube-api-access-zh6vv major:0 minor:442 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:441 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:437 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ac23160-d159-4e38-9a63-691052ae8211/volumes/kubernetes.io~projected/kube-api-access-4jxsw:{mountpoint:/var/lib/kubelet/pods/7ac23160-d159-4e38-9a63-691052ae8211/volumes/kubernetes.io~projected/kube-api-access-4jxsw major:0 minor:420 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/alertmanager/9 major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/config-reloader/6 major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~empty-dir/config-out major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~projected/kube-api-access-f768m:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~projected/kube-api-access-f768m major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~projected/tls-assets major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/config-volume major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/web-config major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volumes/kubernetes.io~projected/kube-api-access-cn7q4:{mountpoint:/var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volumes/kubernetes.io~projected/kube-api-access-cn7q4 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~projected/kube-api-access-mz2b8:{mountpoint:/var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~projected/kube-api-access-mz2b8 major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~projected/kube-api-access-8zsf4:{mountpoint:/var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~projected/kube-api-access-8zsf4 major:0 minor:260 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad0e377c-18f5-45ee-aabf-8b228f379db0/volumes/kubernetes.io~projected/kube-api-access-wgckl:{mountpoint:/var/lib/kubelet/pods/ad0e377c-18f5-45ee-aabf-8b228f379db0/volumes/kubernetes.io~projected/kube-api-access-wgckl major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad0e377c-18f5-45ee-aabf-8b228f379db0/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/ad0e377c-18f5-45ee-aabf-8b228f379db0/volumes/kubernetes.io~secret/metrics-certs major:0 minor:411 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f/volumes/kubernetes.io~projected/kube-api-access-j9zhc:{mountpoint:/var/lib/kubelet/pods/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f/volumes/kubernetes.io~projected/kube-api-access-j9zhc major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce2518fb-0bf8-4628-9927-c46800cb2b3c/volumes/kubernetes.io~projected/kube-api-access-77hn5:{mountpoint:/var/lib/kubelet/pods/ce2518fb-0bf8-4628-9927-c46800cb2b3c/volumes/kubernetes.io~projected/kube-api-access-77hn5 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8/volumes/kubernetes.io~projected/kube-api-access-qmrdn:{mountpoint:/var/lib/kubelet/pods/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8/volumes/kubernetes.io~projected/kube-api-access-qmrdn major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dfd15a52-52ae-4718-a562-ba25778c39ea/volumes/kubernetes.io~projected/kube-api-access-xlvlb:{mountpoint:/var/lib/kubelet/pods/dfd15a52-52ae-4718-a562-ba25778c39ea/volumes/kubernetes.io~projected/kube-api-access-xlvlb major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fd5c8298-1f37-4c24-bb52-1536b0e93028/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/fd5c8298-1f37-4c24-bb52-1536b0e93028/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:415 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565/volumes/kubernetes.io~projected/kube-api-access-fj9zk:{mountpoint:/var/lib/kubelet/pods/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565/volumes/kubernetes.io~projected/kube-api-access-fj9zk major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/dc0c312a36bc2d032a5da1a77462eb61088d1492ceed5efc898dbb75688890a9/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/b9e1b01c7dfd339780066ca8045a1ac96aae8b2de7e417211954c41b68e302ee/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/1a139738c50dec036d78e92021dfb07eabf80e7af98cbde2b06a4fee546a09c7/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/0e0448255b2c7e296a8d214790b4ee82ba190279c64a49ca5cf3f4b9ad3cc5a5/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/a739145d47a8beff271161d21b4ec05989cb628a7cbee27269f3482a3f37a4af/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/4bd0c70246f05d5c616fbe97eec25923f88f1929b6248e2e58ac03b1aa304573/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/dfa55cd9152c7e7093839c279eabafbe5a3d965f61258d81a608a10cb1cb53fd/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/1ce0bd3929e52f77ae424cc3b79df62565233c46fd38e20f6176c1d9c87cca4e/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/ad256c7f732062414439bd61c2f39d82e55bf9512d069efc7696da0525f08792/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/a9159c4ce891e70f71de884eefa48004f96a828f6e43bfa90e96e0ff9d1b11ee/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/13bde8165db805182d9c57a3cdb547e683f9e0cf0d369231e287455028fd87de/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/bd419346f72a4728118a16b4d5694890afdc5e80babf5acf9bd89fe10b53af93/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/9db27f2be10647c3739f1558c5f34fa2b6a381c0b9053d7094e6488bd0c58cd2/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/5efbc20e2242807f6480eb472ab71d327f3a484ab69c257719cadc795b970292/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/234dcef52b6a85708fdf1ff95787775f0be803901e1bcfe4a963c9b380f727d5/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/101b8416b1af79310b934b52ae60179cbd086b33e8fed5499a83cc525fba21b7/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/31c341db1c423793ecdcc623d5f092877ee410ccf28d867c4b8393b15b9c7b3f/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/925cba761fb170d52d4899bf21694ec03afc2ca1d26a4379b0f4374a758c0ec2/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/395003f7378c0a41b99253e1e37cc4c717f531faf8661822a8652af206c851c7/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/555f1ce266ed897bf775d3605901e8847750a84f9d2973e4400afe9139317bca/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/a94ad93708ceb25e06caa4772b48b57db27ccbc220cd6ff0477eea164819be07/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/c9e217ead184cfc81e3cb705280b9c78758eabb9283d676b1c8874158e326961/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/95f81fc10c28f5e3e4d10001484ac4365cd8640360ace36b8236dc4c5fedc6af/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/a4b7c3e36e64fed04fde5384b8426729a7f681061ccdc8f9cc642e395698484d/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/8ec02c401e24ffc9cbd02ee6da95a2bdf9b08711be61a1eb19898b0c2f3b2758/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/fe6c9a599517dd63b5768f9c87ba30d051071845646addff00744fc90e494dca/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/e401b4571983eb2eba229538681e4910f46aa1eb54664ccf219b32a7013c75f0/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/ff368bf809546edbe6e9f7efa6e0e948777dc5488a96727de4d90850575f109d/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/9769e529538c61e41dbba0bda29b73c96b0fdde8502ef6ecafbf97c41db3a2a8/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/ff5772f1e9eb06c01273f2f74e3b53be1a1427ffc6c17f5307c93819600ac9b0/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/eda028f766e65f3a8c95897989c0b0dce0bd0514636e90cd875830609dc77cc7/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/b3ef9938bc79e213eeaae99d09910c0d6b5c2dcfb2583f458d4a041e6f23c409/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/42c7956e93a4198c0031125be1c8820b260259b73f0568a890435f340283e38d/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/a6ac8195c45c4c6c9ba7fe2e7cd64bde752bacdacd352cf08d7ca9e7331ecb08/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/0abbb1c6d6a08c09bcc801956b1a9e9ffa3f8387fd4e72413ce467a862f5630e/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/46e8e2f2a1c045b2c65323ff3f4ba29f86d245ab3a14ed618b37efcc5941832e/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/cf6cf256a04c3c8c8288e37cb56163fb26a6ae7dfc52edd86400884493f9732c/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/4c83d0d78e112128566167de75f054e6ffde21cc2aba74c9d3ff19e8beed25cc/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/881d82ce5c265f1dc9f59e6dd10d34b9925e9f0cc64f95a88a5ac204c3c8ff67/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/efb1c8f1dd455bafddad58cd4054d4739e8351595de1636ef3b8e4e6540b3a74/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-317:{mountpoint:/var/lib/containers/storage/overlay/295faec9607f2d5138532c0bf5e28791d72a357a6c06130e4988fd4e21d2f1aa/merged major:0 minor:317 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/3eb6fd894e3cd6cbdc079db935bb4d502aaf87c65a04fbf32a866843c5519120/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/6cee11a8aa295160e3d7b1ce6b25bb5d6b076db3f608239d137d67cdb160c9ff/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/7afd557bb54a1e982d115b882f2c33afdefe82bf399023ad0705c73ec9e462e6/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/4b396047d57f242c33376834e3ac7262ff39cd57aaeff65e6f9cfb7b69d69716/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/e303065c6649438b0b448a1baa8e1ac75f6d093506896fae235a541a2e252af5/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/45d1942a75e2e221ddfb025c8e7af30303abedf8cd7e2e0bf771cd3282da8806/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/7fbbb0f38efe2bfd3fdf0181cb3ee495adce0a07212b7b73df7135c5954ab3b4/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/80b2aa44eb9a1e8eca32558b0132c8855ba4eef6eda84583caebe78adebb68c3/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-370:{mountpoint:/var/lib/containers/storage/overlay/99cd74000a92a95a94cbab1c27d9d0ae8ec65f315714b47902f1ae5fb1f0a5ca/merged major:0 minor:370 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/ed98fb5646c6c91840c57bf3abe8569dce07a7cbd87526796f750ce13ccb68ae/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/787be428c33d67827d52e6ca04ee6399baa132f593e4a85bd591ecafee49fbfd/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-385:{mountpoint:/var/lib/containers/storage/overlay/0674278e6ed2d2cdce303406d49df0d19c6858ec539ab2597907527de93a4081/merged major:0 minor:385 fsType:overlay blockSize:0} overlay_0-390:{mountpoint:/var/lib/containers/storage/overlay/c9abb0c05d032a7488f352b8ec72e258f4ddd9c00792059abf6132e50b07200d/merged major:0 minor:390 fsType:overlay blockSize:0} overlay_0-395:{mountpoint:/var/lib/containers/storage/overlay/9ccce6e6f4b8248cbb84d6c23db500b0ce84234270d90ec3acad9e70b249dd34/merged major:0 minor:395 fsType:overlay blockSize:0} overlay_0-400:{mountpoint:/var/lib/containers/storage/overlay/268a4434c995808752ca0bc094b3f770c04eb43e0e64e212279dcadf5ed2ab35/merged major:0 minor:400 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/5a6153c3317517e44ef203e0f5b6daa0321d20270729137dfc96d7745fb53945/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/ecfc87a2cfb12188b8aecfe9df554af942f2a268a1652fac72d42be31e512276/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/c5b4140c1f865ad950ccd8f0064deb0cc5ac2f3673d9f219ede1cd4b9b592a72/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-427:{mountpoint:/var/lib/containers/storage/overlay/b42f9c14abdcf471d0814a1a1b811fa3964525cd97aea4ad1140de34f2950f7b/merged major:0 minor:427 fsType:overlay blockSize:0} overlay_0-429:{mountpoint:/var/lib/containers/storage/overlay/e96440e71877e6af8ceaa98fe80a40898d77462147694122036d3433fcbc3d4a/merged major:0 minor:429 fsType:overlay blockSize:0} overlay_0-431:{mountpoint:/var/lib/containers/storage/overlay/02f46afe4554cd0e7559433ee4fa4b3b1f083d4c4204f49c38777d55237b055e/merged major:0 minor:431 fsType:overlay blockSize:0} overlay_0-445:{mountpoint:/var/lib/containers/storage/overlay/0d8a45cd3a9bb9dc7a02afe411e7f92062fb1ab395d1adcc640c2caf1d3497a7/merged major:0 minor:445 fsType:overlay blockSize:0} overlay_0-447:{mountpoint:/var/lib/containers/storage/overlay/c460629f7a343e6c1b849f4de8e5b6c384c14b75bf2fe0b119c398a27bcb9798/merged major:0 minor:447 fsType:overlay blockSize:0} overlay_0-449:{mountpoint:/var/lib/containers/storage/overlay/cc483d512e85c3c8d8690b199ae24ce949edb6ee10a05902270ace2e5780a7ea/merged major:0 minor:449 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/6ea20a08ddf73e14a8eb3bd11a58bec64631690201503fc4a74a62476635bc9d/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/cc7adec74dee0138e7b76de0ab2d74d295f48464b8e07479b4a62212ed3fa9af/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/db6bad03ffb69468b21c4a34221917d08918163ce272e2b2e4d8b5cc399ff1a6/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/79677ec45cff18fadb56c497fe5cc1501343248764c8c7c7c97adff1f727ada6/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/2fadcd0c44c8f72b9cfb7bbec847f771cbda51e1a914b7de6ca65917ed7c6458/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/8944647bb18574bfc20aa52b53bd3d616b25fa7edec81915de74b92816f160e5/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/3bb9a9ea4f2ee5890df18dd1378f6e0ad03d932a856a42a1e359bcf5e34b06d2/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/f496d78240566c2184d53bdcceefb52de654990ac0efaf1895683c604de474b1/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-93:{mountpoint:/var/lib/containers/storage/overlay/b695531b45d3faf46f73db7bddc5cdd6fb6b2d05343638554f62240455553223/merged major:0 minor:93 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/f1612668510c51657b23e9f7a998c4190a7d9594d321a6e1f961671a47b92dea/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.629570    5647 manager.go:217] Machine: {Timestamp:2026-11-03 22:22:56.628981851 +0000 UTC m=+0.069519101 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100133 MemoryCapacity:32812003328 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ceb707cbc1d7c7df0dbc3ea47a0fe SystemUUID:ec2ceb70-7cbc-1d7c-7df0-dbc3ea47a0fe BootID:9819344e-aed4-4147-a393-9b34f03f4c0d Filesystems:[{Device:/var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ce34cb7a203574d699b19f13cf706748ce59636979d65a0705b47e90cc44d813/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-93 DeviceMajor:0 DeviceMinor:93 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:257 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:293 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/1f6b6450-cd73-4506-85d1-94ef030fe583/volumes/kubernetes.io~projected/kube-api-access-zsjsb DeviceMajor:0 DeviceMinor:54 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/80af86e253ef2ce44fd81537e34b5fd7fcb82251254ba410abad1e947a7c62c0/userdata/shm DeviceMajor:0 DeviceMinor:249 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/ce2518fb-0bf8-4628-9927-c46800cb2b3c/volumes/kubernetes.io~projected/kube-api-access-77hn5 DeviceMajor:0 DeviceMinor:55 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565/volumes/kubernetes.io~projected/kube-api-access-fj9zk DeviceMajor:0 DeviceMinor:197 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:295 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/485ddda6-f359-4513-9a89-239467e9f03a/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:251 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-317 DeviceMajor:0 DeviceMinor:317 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~projected/kube-api-access-nhw5z DeviceMajor:0 DeviceMinor:243 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:290 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:295 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/39d8576dbab5cc662bb4c00dd3439a07053bc31b80ef03dcfbec8048d0a8bbd1/userdata/shm DeviceMajor:0 DeviceMinor:418 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:298 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:323 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-370 DeviceMajor:0 DeviceMinor:370 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16405999616 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:437 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/485ddda6-f359-4513-9a89-239467e9f03a/volumes/kubernetes.io~projected/kube-api-access-vkhzs DeviceMajor:0 DeviceMinor:206 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:256 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/5db69683-41ca-4f88-a7ab-5a1174170725/volumes/kubernetes.io~projected/kube-api-access-ckdgq DeviceMajor:0 DeviceMinor:259 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/56109620006a9d0e0aea12cf87f4414b5d3370ebbc6ca80ff5d6eb72f17de557/userdata/shm DeviceMajor:0 DeviceMinor:342 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-385 DeviceMajor:0 DeviceMinor:385 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ad0e377c-18f5-45ee-aabf-8b228f379db0/volumes/kubernetes.io~projected/kube-api-access-wgckl DeviceMajor:0 DeviceMinor:58 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:286 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:325 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3b00ce6e5e5dc82bd9429956df4ea8a97fadea907ef3efce57e2974c925cd8f1/userdata/shm DeviceMajor:0 DeviceMinor:443 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/af2f0fbaa530a742678f3ac999deb609fcafc4da133319a95deb6b099b2865ee/userdata/shm DeviceMajor:0 DeviceMinor:71 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:242 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:326 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:338 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:295 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-447 DeviceMajor:0 DeviceMinor:447 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4bd65c82c3fd805ef13820b6b0cd2447e2b385c4e9a40237e22fd8651bed0d66/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~projected/kube-api-access-8zsf4 DeviceMajor:0 DeviceMinor:260 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:340 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/17f89a2e3a273ff2a67baf7c40109cf5229c488a5d7f8e295931f2d2f7bd8154/userdata/shm DeviceMajor:0 DeviceMinor:263 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:292 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/395f30f1-f5f2-4164-90bd-02bfbaca7d31/volumes/kubernetes.io~projected/kube-api-access-9z6th DeviceMajor:0 DeviceMinor:205 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f/volumes/kubernetes.io~projected/kube-api-access-j9zhc DeviceMajor:0 DeviceMinor:56 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ebe5c6dd23089ed9d8463b4e06475b271939884066a6d276d54070a5168eab99/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~projected/kube-api-access-f768m DeviceMajor:0 DeviceMinor:299 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:324 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:339 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-431 DeviceMajor:0 DeviceMinor:431 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~projected/kube-api-access-zh6vv DeviceMajor:0 DeviceMinor:442 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volumes/kubernetes.io~projected/kube-api-access-cn7q4 DeviceMajor:0 DeviceMinor:59 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:287 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/79988fe6f2d4e25bf75a438d7305fcc6444424973573ce3f0cb1475e2a0e9c83/userdata/shm DeviceMajor:0 DeviceMinor:332 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-395 DeviceMajor:0 DeviceMinor:395 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ac23160-d159-4e38-9a63-691052ae8211/volumes/kubernetes.io~projected/kube-api-access-4jxsw DeviceMajor:0 DeviceMinor:420 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/10ee490c492330638c8e951061b789a4aa3b7c2a65dfc99cf01239b14f512404/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:441 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7f2dcccb375531f6bd24065f89da9ba70cdaf57ebbda802883fcbbec86e647e1/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c85e3f598d7a55956de744ce87deb19c74014854ac37926cfcab96feddeb9146/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-445 DeviceMajor:0 DeviceMinor:445 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/843867e247b0c2a74e0f38cc87e1f4d8da0d1f0376d0d46ace77307cfcbcc5de/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:295 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/395f30f1-f5f2-4164-90bd-02bfbaca7d31/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:202 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~projected/kube-api-access-92z8d DeviceMajor:0 DeviceMinor:328 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dfd15a52-52ae-4718-a562-ba25778c39ea/volumes/kubernetes.io~projected/kube-api-access-xlvlb DeviceMajor:0 DeviceMinor:51 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-390 DeviceMajor:0 DeviceMinor:390 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c944860f941df63d4d1b46a3a3e724fa8faa575e38245951d80f81bc02cf322b/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fd5c8298-1f37-4c24-bb52-1536b0e93028/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:415 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/75258429ddbec7b2623724af3756989561043beb1332135c4f9f0d1eaa57ecc2/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5c6cf1320dc0c5db26970952d43e0fd9a8f8893c2c3655dfcf489d266206bcd8/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-400 DeviceMajor:0 DeviceMinor:400 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-429 DeviceMajor:0 DeviceMinor:429 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8308da23-cdc6-4e50-91a2-9b158e17ca3b/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:47 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/6d457444-8760-482f-a08d-7b20024db9fa/volumes/kubernetes.io~projected/kube-api-access-s49km DeviceMajor:0 DeviceMinor:57 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:294 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:290 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-449 DeviceMajor:0 DeviceMinor:449 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/01fe56b3-4ca9-416a-8480-e199577ac2d5/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:241 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/4196c457-6d8d-4a01-b299-653d0b4e4445/volumes/kubernetes.io~projected/kube-api-access-9mvqp DeviceMajor:0 DeviceMinor:341 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/8afabbf8-4122-42aa-95c6-75e6d41bf40d/volumes/kubernetes.io~projected/kube-api-access-mz2b8 DeviceMajor:0 DeviceMinor:52 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a26ae6a44a25759ab38d917b596be6f63ade0ef65561db450fb7468662e5717e/userdata/shm DeviceMajor:0 DeviceMinor:303 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-427 DeviceMajor:0 DeviceMinor:427 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f03fcb31d36e5fc094fa5beb818c5b0a4266aaf72071b2deef43e60e7cc4a91c/userdata/shm DeviceMajor:0 DeviceMinor:261 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:291 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5c02c6a23a3905c608facba35481a509275b5631d39f6356c6d73378f445ca63/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5492a31637d4407de7b75d2d3299f777b16d5325f1459be26d5b2cbad764207a/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e91ab02-2291-4937-b729-0422ee58256b/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:258 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:327 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3b22f90c3432e8bfe0f733b5962a010f658c099106a9042cd41fcebce903bb53/userdata/shm DeviceMajor:0 DeviceMinor:416 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c6fe8e5169818a54cf1ac46ad312e7b1d5aa6e84767dd98e5227f1be59e84c5a/userdata/shm DeviceMajor:0 DeviceMinor:421 Capacity:67108864 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/0b0deac8-c535-4d85-b27e-968b22b55c14/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8/volumes/kubernetes.io~projected/kube-api-access-qmrdn DeviceMajor:0 DeviceMinor:53 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/3ead9dc1-5785-432c-a28d-6dd002c4dc27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:322 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true} {Device:/var/lib/kubelet/pods/ad0e377c-18f5-45ee-aabf-8b228f379db0/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:411 Capacity:31633403904 Type:vfs Inodes:4005371 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:39d8576dbab5cc6 MacAddress:5e:ea:aa:28:8a:c1 Speed:10000 Mtu:8901} {Name:3b00ce6e5e5dc82 MacAddress:82:2e:06:cd:62:43 Speed:10000 Mtu:8901} {Name:3b22f90c3432e8b MacAddress:de:18:e6:3b:67:96 Speed:10000 Mtu:8901} {Name:56109620006a9d0 MacAddress:02:0d:48:d8:5f:35 Speed:10000 Mtu:8901} {Name:5c6cf1320dc0c5d MacAddress:02:a4:4a:65:f4:2d Speed:10000 Mtu:8901} {Name:79988fe6f2d4e25 MacAddress:fa:98:45:1b:6a:91 Speed:10000 Mtu:8901} {Name:7f2dcccb375531f MacAddress:9e:f0:cf:1b:88:f6 Speed:10000 Mtu:8901} {Name:80af86e253ef2ce MacAddress:46:10:6b:c5:29:e8 Speed:10000 Mtu:8901} {Name:a26ae6a44a25759 MacAddress:b2:1c:97:d0:ec:74 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:79:fd:a5:68:ed Speed:0 Mtu:9001} {Name:br-int MacAddress:ca:20:7a:b7:3a:16 Speed:0 Mtu:8901} {Name:c6fe8e5169818a5 MacAddress:9a:6e:14:15:db:ee Speed:10000 Mtu:8901} {Name:c85e3f598d7a559 MacAddress:7a:28:f2:f0:f6:89 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:79:fd:a5:68:ed Speed:0 Mtu:9001} {Name:f03fcb31d36e5fc MacAddress:96:2b:f9:02:ee:f9 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:fe:c5:a3:6f:c2:53 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ea:a1:b2:5b:e2:8e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812003328 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630068    5647 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630120    5647 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630494    5647 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630533    5647 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-187.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630728    5647 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630741    5647 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630758    5647 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630779    5647 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.630858    5647 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.631100    5647 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.631239    5647 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.631280    5647 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.631292    5647 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.631307    5647 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.631325    5647 kubelet.go:405] "Adding apiserver pod source"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.631337    5647 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.633166    5647 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.633193    5647 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 22:22:56.639853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.633798    5647 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 22:22:56.644519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.644502    5647 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 22:22:56.644519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.644519    5647 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 22:22:56.644646 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.644636    5647 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 03 22:22:56.644873 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.644860    5647 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 22:22:56.644924 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.644891    5647 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 22:22:56.645006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.644996    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 22:22:56.645052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645012    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 22:22:56.645052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645022    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 22:22:56.645052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645032    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 22:22:56.645052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645040    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 22:22:56.645052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645047    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 22:22:56.645052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645052    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 22:22:56.645254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645058    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 22:22:56.645254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645065    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 22:22:56.645254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645071    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 22:22:56.645254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645086    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 22:22:56.645254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645095    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 22:22:56.645254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645113    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 22:22:56.645254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.645119    5647 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 22:22:56.648886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.648864    5647 server.go:1263] "Started kubelet"
Nov 03 22:22:56.649100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.649050    5647 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 22:22:56.649157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.649123    5647 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 22:22:56.649157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.649138    5647 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 22:22:56.649402 ip-10-0-132-187 systemd[1]: Started Kubernetes Kubelet.
Nov 03 22:22:56.650200 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.649507    5647 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 22:22:56.650483 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.650466    5647 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 22:22:56.652408 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.652392    5647 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 22:22:56.652651 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.652603    5647 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.653692    5647 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.653710    5647 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:56.653928    5647 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-187.us-west-1.compute.internal\" not found"
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.654551    5647 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.654534    5647 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 22:16:46 +0000 UTC" deadline="2028-05-28 08:10:46.269966003 +0000 UTC"
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.654572    5647 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13713h47m49.615397379s"
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.655031    5647 factory.go:55] Registering systemd factory
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.655050    5647 factory.go:223] Registration of the systemd container factory successfully
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.657945    5647 factory.go:153] Registering CRI-O factory
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.657961    5647 factory.go:223] Registration of the crio container factory successfully
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658017    5647 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658052    5647 factory.go:103] Registering Raw factory
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658077    5647 manager.go:1199] Started watching for new ooms in manager
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658308    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db69683-41ca-4f88-a7ab-5a1174170725" volumeName="kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-tls" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658343    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658359    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e91ab02-2291-4937-b729-0422ee58256b" volumeName="kubernetes.io/empty-dir/9e91ab02-2291-4937-b729-0422ee58256b-volume-directive-shadow" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658375    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad0e377c-18f5-45ee-aabf-8b228f379db0" volumeName="kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658407    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe23ba2a-adba-4d25-b08c-cbcd0d0c0565" volumeName="kubernetes.io/secret/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-metrics-tls" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658424    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01fe56b3-4ca9-416a-8480-e199577ac2d5" volumeName="kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658442    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="395f30f1-f5f2-4164-90bd-02bfbaca7d31" volumeName="kubernetes.io/configmap/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658452    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658468    5647 manager.go:319] Starting recovery of all containers
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658466    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8308da23-cdc6-4e50-91a2-9b158e17ca3b" volumeName="kubernetes.io/secret/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658552    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01fe56b3-4ca9-416a-8480-e199577ac2d5" volumeName="kubernetes.io/configmap/01fe56b3-4ca9-416a-8480-e199577ac2d5-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658576    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-tls" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658597    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="485ddda6-f359-4513-9a89-239467e9f03a" volumeName="kubernetes.io/secret/485ddda6-f359-4513-9a89-239467e9f03a-cert" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658612    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" volumeName="kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658635    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8308da23-cdc6-4e50-91a2-9b158e17ca3b" volumeName="kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-script-lib" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658659    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8308da23-cdc6-4e50-91a2-9b158e17ca3b" volumeName="kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-config" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658693    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8afabbf8-4122-42aa-95c6-75e6d41bf40d" volumeName="kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-tmp" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658714    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfd15a52-52ae-4718-a562-ba25778c39ea" volumeName="kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-cni-binary-copy" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658728    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="395f30f1-f5f2-4164-90bd-02bfbaca7d31" volumeName="kubernetes.io/projected/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-api-access-9z6th" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658746    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="395f30f1-f5f2-4164-90bd-02bfbaca7d31" volumeName="kubernetes.io/secret/395f30f1-f5f2-4164-90bd-02bfbaca7d31-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658760    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658776    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4196c457-6d8d-4a01-b299-653d0b4e4445" volumeName="kubernetes.io/empty-dir/4196c457-6d8d-4a01-b299-653d0b4e4445-audit-log" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658794    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db69683-41ca-4f88-a7ab-5a1174170725" volumeName="kubernetes.io/projected/5db69683-41ca-4f88-a7ab-5a1174170725-kube-api-access-ckdgq" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658806    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d457444-8760-482f-a08d-7b20024db9fa" volumeName="kubernetes.io/projected/6d457444-8760-482f-a08d-7b20024db9fa-kube-api-access-s49km" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658822    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" volumeName="kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658838    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe23ba2a-adba-4d25-b08c-cbcd0d0c0565" volumeName="kubernetes.io/configmap/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-config-volume" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658854    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658870    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" volumeName="kubernetes.io/projected/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-kube-api-access-j9zhc" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658883    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe23ba2a-adba-4d25-b08c-cbcd0d0c0565" volumeName="kubernetes.io/empty-dir/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-tmp-dir" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658906    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db69683-41ca-4f88-a7ab-5a1174170725" volumeName="kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658924    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db69683-41ca-4f88-a7ab-5a1174170725" volumeName="kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658938    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" volumeName="kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658959    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.658980    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2518fb-0bf8-4628-9927-c46800cb2b3c" volumeName="kubernetes.io/projected/ce2518fb-0bf8-4628-9927-c46800cb2b3c-kube-api-access-77hn5" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660082    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe23ba2a-adba-4d25-b08c-cbcd0d0c0565" volumeName="kubernetes.io/projected/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-kube-api-access-fj9zk" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660113    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b0deac8-c535-4d85-b27e-968b22b55c14" volumeName="kubernetes.io/configmap/0b0deac8-c535-4d85-b27e-968b22b55c14-konnectivity-ca" seLinuxMountContext=""
Nov 03 22:22:56.660106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660127    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b0deac8-c535-4d85-b27e-968b22b55c14" volumeName="kubernetes.io/secret/0b0deac8-c535-4d85-b27e-968b22b55c14-agent-certs" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660161    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8afabbf8-4122-42aa-95c6-75e6d41bf40d" volumeName="kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-tuned" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660180    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfd15a52-52ae-4718-a562-ba25778c39ea" volumeName="kubernetes.io/projected/dfd15a52-52ae-4718-a562-ba25778c39ea-kube-api-access-xlvlb" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660194    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4196c457-6d8d-4a01-b299-653d0b4e4445" volumeName="kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-client-ca-bundle" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660213    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd5c8298-1f37-4c24-bb52-1536b0e93028" volumeName="kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660226    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4196c457-6d8d-4a01-b299-653d0b4e4445" volumeName="kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-metrics-server-audit-profiles" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660243    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4196c457-6d8d-4a01-b299-653d0b4e4445" volumeName="kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-client-certs" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660258    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660288    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8308da23-cdc6-4e50-91a2-9b158e17ca3b" volumeName="kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-env-overrides" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660304    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8afabbf8-4122-42aa-95c6-75e6d41bf40d" volumeName="kubernetes.io/projected/8afabbf8-4122-42aa-95c6-75e6d41bf40d-kube-api-access-mz2b8" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660320    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8" volumeName="kubernetes.io/projected/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kube-api-access-qmrdn" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660338    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01fe56b3-4ca9-416a-8480-e199577ac2d5" volumeName="kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-tls" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660351    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f6b6450-cd73-4506-85d1-94ef030fe583" volumeName="kubernetes.io/projected/1f6b6450-cd73-4506-85d1-94ef030fe583-kube-api-access-zsjsb" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660369    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/projected/3ead9dc1-5785-432c-a28d-6dd002c4dc27-kube-api-access-92z8d" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660414    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" volumeName="kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660434    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" volumeName="kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660448    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfd15a52-52ae-4718-a562-ba25778c39ea" volumeName="kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-daemon-config" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660462    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/empty-dir/3ead9dc1-5785-432c-a28d-6dd002c4dc27-tmp" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660480    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660493    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db69683-41ca-4f88-a7ab-5a1174170725" volumeName="kubernetes.io/empty-dir/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-textfile" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660511    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660546    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" volumeName="kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660569    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01fe56b3-4ca9-416a-8480-e199577ac2d5" volumeName="kubernetes.io/projected/01fe56b3-4ca9-416a-8480-e199577ac2d5-kube-api-access-nhw5z" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660600    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/configmap/3ead9dc1-5785-432c-a28d-6dd002c4dc27-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660615    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660629    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="485ddda6-f359-4513-9a89-239467e9f03a" volumeName="kubernetes.io/projected/485ddda6-f359-4513-9a89-239467e9f03a-kube-api-access-vkhzs" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660648    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" volumeName="kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660661    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" volumeName="kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660717    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" volumeName="kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660730    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ac23160-d159-4e38-9a63-691052ae8211" volumeName="kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660746    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f6b6450-cd73-4506-85d1-94ef030fe583" volumeName="kubernetes.io/configmap/1f6b6450-cd73-4506-85d1-94ef030fe583-iptables-alerter-script" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660787    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660801    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660819    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660833    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8308da23-cdc6-4e50-91a2-9b158e17ca3b" volumeName="kubernetes.io/projected/8308da23-cdc6-4e50-91a2-9b158e17ca3b-kube-api-access-cn7q4" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660847    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e91ab02-2291-4937-b729-0422ee58256b" volumeName="kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-metrics-client-ca" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660865    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e91ab02-2291-4937-b729-0422ee58256b" volumeName="kubernetes.io/projected/9e91ab02-2291-4937-b729-0422ee58256b-kube-api-access-8zsf4" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.660878    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e91ab02-2291-4937-b729-0422ee58256b" volumeName="kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661504    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4196c457-6d8d-4a01-b299-653d0b4e4445" volumeName="kubernetes.io/projected/4196c457-6d8d-4a01-b299-653d0b4e4445-kube-api-access-9mvqp" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661530    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e91ab02-2291-4937-b729-0422ee58256b" volumeName="kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-tls" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661544    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d457444-8760-482f-a08d-7b20024db9fa" volumeName="kubernetes.io/empty-dir/6d457444-8760-482f-a08d-7b20024db9fa-tmp-dir" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661557    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661569    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661583    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad0e377c-18f5-45ee-aabf-8b228f379db0" volumeName="kubernetes.io/projected/ad0e377c-18f5-45ee-aabf-8b228f379db0-kube-api-access-wgckl" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661594    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" volumeName="kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-binary-copy" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661607    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="395f30f1-f5f2-4164-90bd-02bfbaca7d31" volumeName="kubernetes.io/empty-dir/395f30f1-f5f2-4164-90bd-02bfbaca7d31-data-volume" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661618    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-grpc-tls" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661629    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4196c457-6d8d-4a01-b299-653d0b4e4445" volumeName="kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661640    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db69683-41ca-4f88-a7ab-5a1174170725" volumeName="kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661652    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661681    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ea0760a-ec14-4dce-b11b-d746d72de9ee" volumeName="kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661693    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e91ab02-2291-4937-b729-0422ee58256b" volumeName="kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661705    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce2518fb-0bf8-4628-9927-c46800cb2b3c" volumeName="kubernetes.io/configmap/ce2518fb-0bf8-4628-9927-c46800cb2b3c-serviceca" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661724    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ead9dc1-5785-432c-a28d-6dd002c4dc27" volumeName="kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661735    5647 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4196c457-6d8d-4a01-b299-653d0b4e4445" volumeName="kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-tls" seLinuxMountContext=""
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661745    5647 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 22:22:56.662367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.661752    5647 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 22:22:56.684248 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.683858    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:22:56.684415 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.684307    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:22:56.685790 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.685737    5647 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 22:22:56.688128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.688108    5647 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 22:22:56.688128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.688130    5647 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 22:22:56.688266 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.688152    5647 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 22:22:56.688266 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:56.688195    5647 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 03 22:22:56.719705 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.719642    5647 generic.go:356] "Generic (PLEG): container finished" podID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerID="0f11b7165af4bc6c0c544820b2a21aa30ecc72f833193c1e810261b33cc3a926" exitCode=0
Nov 03 22:22:56.724690 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.724613    5647 generic.go:356] "Generic (PLEG): container finished" podID="46184df3bcae45c6ec28abb5e9a3b973" containerID="6f849f6e327ff30960c8f8b21fdeb4402f689a02fdb897456a86cb4064d04b34" exitCode=0
Nov 03 22:22:56.729074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.729051    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:22:56.729418 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.729399    5647 generic.go:356] "Generic (PLEG): container finished" podID="8308da23-cdc6-4e50-91a2-9b158e17ca3b" containerID="b585e70638cb9743c8b29b916681f84f7105911e70bd6e25c8ccccc2adf8da5b" exitCode=1
Nov 03 22:22:56.752457 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.752371    5647 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="cd0a5df0dbb5290c6042ae0c497044453d297295ab97899a9f93ee710ac386a9" exitCode=0
Nov 03 22:22:56.752457 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.752403    5647 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="2fdf94163c19c6d3e2c79bc61dd6e5eb16f3b08eb555201cd213ebb0969c0375" exitCode=0
Nov 03 22:22:56.752457 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.752410    5647 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="ce9f5a4aced8a190940e2c1852b53755cae978d86db479955e18dce91e710d51" exitCode=0
Nov 03 22:22:56.752457 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.752418    5647 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="68ce21292266f0898ead8b129c42f567f731ec3c7505307a2f954d0e77c45167" exitCode=0
Nov 03 22:22:56.752457 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.752426    5647 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="abad503915913b63cfc8194a02b913c0239a930ad7338a27d4ff170200cc7dda" exitCode=0
Nov 03 22:22:56.752457 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.752434    5647 generic.go:356] "Generic (PLEG): container finished" podID="b9f62230-4ed4-49ec-bc29-a69f0b46bd8f" containerID="5103f8e2da6d23a89e795d849335d0fdb5c54b67b1bdc733b6f399b50dea129e" exitCode=0
Nov 03 22:22:56.754216 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:56.754190    5647 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-187.us-west-1.compute.internal\" not found"
Nov 03 22:22:56.762029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.761952    5647 generic.go:356] "Generic (PLEG): container finished" podID="5db69683-41ca-4f88-a7ab-5a1174170725" containerID="319e36fc27aa84d5ec18bb03473e16e24a664fb9b7ca0947d3ec23c716eb1301" exitCode=0
Nov 03 22:22:56.782944 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.782919    5647 manager.go:324] Recovery completed
Nov 03 22:22:56.788292 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:56.788251    5647 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 03 22:22:56.809533 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.809509    5647 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:56.811662 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.811634    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:56.811770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.811696    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:56.811770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.811713    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:56.814638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814615    5647 cpu_manager.go:226] "Starting" policy="none"
Nov 03 22:22:56.814638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814634    5647 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:56.814870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814635    5647 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 22:22:56.814870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814710    5647 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 22:22:56.814953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814880    5647 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 03 22:22:56.814953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814895    5647 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 03 22:22:56.814953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814913    5647 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 03 22:22:56.814953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814918    5647 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 03 22:22:56.814953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814924    5647 policy_none.go:50] "Start"
Nov 03 22:22:56.814953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814934    5647 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 22:22:56.814953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.814944    5647 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 22:22:56.815905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.815887    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:56.816000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.815925    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:56.816000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.815935    5647 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 03 22:22:56.816000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.815951    5647 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 03 22:22:56.816000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.815959    5647 policy_none.go:44] "Start"
Nov 03 22:22:56.816153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.815939    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:56.821308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.821275    5647 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 22:22:56.825663 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.825647    5647 manager.go:341] "Starting Device Plugin manager"
Nov 03 22:22:56.825762 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:56.825700    5647 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 22:22:56.825762 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.825713    5647 server.go:87] "Starting device plugin registration server"
Nov 03 22:22:56.826007 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.825991    5647 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 22:22:56.826071 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.826011    5647 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 22:22:56.827436 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.827419    5647 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 22:22:56.827544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.827514    5647 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 22:22:56.827598 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.827550    5647 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 22:22:56.827598 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.827557    5647 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 22:22:56.828347 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.827633    5647 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T22:22:56.827540196Z","UUID":"b729c202-0903-4bf4-b58e-117a3211b31f","Handler":null,"Name":"","Endpoint":""}
Nov 03 22:22:56.829833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.829820    5647 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 22:22:56.829951 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.829941    5647 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 22:22:56.830743 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:56.830725    5647 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 22:22:56.830822 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:56.830768    5647 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-187.us-west-1.compute.internal\" not found"
Nov 03 22:22:56.847186 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.847163    5647 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:22:56.847688 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.847622    5647 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:22:56.847862 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.847703    5647 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 03 22:22:56.896922 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.896868    5647 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:22:56.929508 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.929476    5647 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 22:22:56.930906 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.930877    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 22:22:56.931045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.930922    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 22:22:56.931045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.930933    5647 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-187.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 22:22:56.931045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.930984    5647 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:56.945708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.945683    5647 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 22:22:56.988934 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.988864    5647 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"]
Nov 03 22:22:56.989746 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.989722    5647 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:56.989746 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.989714    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"46184df3bcae45c6ec28abb5e9a3b973","Type":"ContainerStarted","Data":"db8f1c6ae76bedd85ccae941e5ee6c5683270ea92bb3c94bdd2ac2a071f33c54"}
Nov 03 22:22:56.989920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.989767    5647 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:56.989920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.989774    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"46184df3bcae45c6ec28abb5e9a3b973","Type":"ContainerDied","Data":"6f849f6e327ff30960c8f8b21fdeb4402f689a02fdb897456a86cb4064d04b34"}
Nov 03 22:22:56.989920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.989792    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"46184df3bcae45c6ec28abb5e9a3b973","Type":"ContainerStarted","Data":"c944860f941df63d4d1b46a3a3e724fa8faa575e38245951d80f81bc02cf322b"}
Nov 03 22:22:56.989920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.989852    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"9cc67bfe6e75d748c4abbd6f6df49a40","Type":"ContainerStarted","Data":"0acbab6299724cd0ebd6d25c163963ceb43d8cb3d6fc81886dd67bb3abf00c13"}
Nov 03 22:22:56.989920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:56.989865    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal" event={"ID":"9cc67bfe6e75d748c4abbd6f6df49a40","Type":"ContainerStarted","Data":"10ee490c492330638c8e951061b789a4aa3b7c2a65dfc99cf01239b14f512404"}
Nov 03 22:22:57.041227 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.041150    5647 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.041379 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.041234    5647 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.048271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.048248    5647 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:22:57.048388 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:57.048325    5647 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.048641 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.048624    5647 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:22:57.048712 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:57.048663    5647 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.148944 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.148905    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.149095 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.148955    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.149095 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.148985    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cc67bfe6e75d748c4abbd6f6df49a40-config\") pod \"kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"9cc67bfe6e75d748c4abbd6f6df49a40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.249434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.249399    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.249624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.249449    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cc67bfe6e75d748c4abbd6f6df49a40-config\") pod \"kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"9cc67bfe6e75d748c4abbd6f6df49a40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.249624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.249475    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.249624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.249490    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.249624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.249521    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/46184df3bcae45c6ec28abb5e9a3b973-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"46184df3bcae45c6ec28abb5e9a3b973\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.249624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.249552    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cc67bfe6e75d748c4abbd6f6df49a40-config\") pod \"kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal\" (UID: \"9cc67bfe6e75d748c4abbd6f6df49a40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.632480 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.632435    5647 apiserver.go:52] "Watching apiserver"
Nov 03 22:22:57.735683 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.735645    5647 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 22:22:57.736283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.736261    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-g9t4g","openshift-dns/dns-default-9fnc7","openshift-image-registry/node-ca-84hzv","openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7","openshift-cluster-node-tuning-operator/tuned-2lfgw","openshift-ovn-kubernetes/ovnkube-node-d7l9z","openshift-ingress-canary/ingress-canary-z2ljt","openshift-multus/multus-rkgdk","openshift-console/console-59c56bd6bd-82w6k","kube-system/konnectivity-agent-qj6mk","openshift-multus/network-metrics-daemon-vr2lh","openshift-network-operator/iptables-alerter-lqjh7","openshift-dns/node-resolver-pvgnl","openshift-monitoring/alertmanager-main-0","openshift-monitoring/node-exporter-8rxz5","kube-system/global-pull-secret-syncer-zhsp8","openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal","openshift-monitoring/metrics-server-6ddc555bd7-sljjr","openshift-insights/insights-runtime-extractor-lmz29","kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal","openshift-monitoring/thanos-querier-b46bc6cb-p4dg9","openshift-multus/multus-additional-cni-plugins-8ck4c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"]
Nov 03 22:22:57.756498 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.756465    5647 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 22:22:57.774406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.774378    5647 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.774633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.774497    5647 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.800574 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.800540    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.800977 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.800948    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.801092 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.800961    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.801092 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801053    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.801092 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801083    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801120    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801124    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801126    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801147    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801136    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801161    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801167    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801214    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801215    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801242    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801249    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 22:22:57.801273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801251    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.801717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801284    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.801717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801291    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 22:22:57.801717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801296    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 22:22:57.801717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801308    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.801717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801392    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 22:22:57.801717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801406    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.801717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.801468    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 22:22:57.808588 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808558    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.808588 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808585    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 22:22:57.808781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808621    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 22:22:57.808781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808723    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 22:22:57.808881 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808786    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ft6gs\""
Nov 03 22:22:57.808881 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808827    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 22:22:57.808881 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808858    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xglhz\""
Nov 03 22:22:57.808990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808909    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 22:22:57.808990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808974    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q82l2\""
Nov 03 22:22:57.809066 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.808995    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-px8pz\""
Nov 03 22:22:57.809066 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809040    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-rrf2w\""
Nov 03 22:22:57.809066 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809064    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b2scz\""
Nov 03 22:22:57.809180 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809114    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mx976\""
Nov 03 22:22:57.809229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809181    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dbbtb\""
Nov 03 22:22:57.809277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809233    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-dx5sz\""
Nov 03 22:22:57.809277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809241    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-j25tw\""
Nov 03 22:22:57.809353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809315    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 22:22:57.809384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809359    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dnsfs\""
Nov 03 22:22:57.809432 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809400    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zhswt\""
Nov 03 22:22:57.809466 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.809451    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m2wh6\""
Nov 03 22:22:57.831590 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831562    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.831744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831600    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.831744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831615    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 03 22:22:57.831744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831622    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 22:22:57.831744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831698    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 22:22:57.831965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831749    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 22:22:57.831965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831779    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 22:22:57.831965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831811    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.831965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831816    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 22:22:57.831965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831874    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vc5c5\""
Nov 03 22:22:57.831965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831925    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.831965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.831961    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-frpm5\""
Nov 03 22:22:57.832187 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.832030    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-94lr2\""
Nov 03 22:22:57.840993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.840973    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 03 22:22:57.841185 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841167    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.841243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841213    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.841360 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841346    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 22:22:57.841437 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841393    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 22:22:57.841437 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841394    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 03 22:22:57.841535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841473    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 03 22:22:57.841535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841478    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 03 22:22:57.841535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841486    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 22:22:57.841535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841520    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 03 22:22:57.841535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841521    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 03 22:22:57.841535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.841522    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 22:22:57.850651 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850628    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 22:22:57.850755 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850652    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 03 22:22:57.850810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850788    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 03 22:22:57.850810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850794    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 03 22:22:57.850921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850810    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 03 22:22:57.850921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850850    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 03 22:22:57.850921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850862    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 03 22:22:57.850921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850864    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-84pw6\""
Nov 03 22:22:57.850921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850891    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 03 22:22:57.850921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850906    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 03 22:22:57.850921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850921    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850955    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850960    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850966    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850928    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850971    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rfslm\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.850961    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.851006    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.851030    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-z56st\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.851045    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q5zlz\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.851053    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.851057    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.851087    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 03 22:22:57.851212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.851093    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-14p5jl2nv912m\""
Nov 03 22:22:57.852439 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852419    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:57.852511 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852453    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.852511 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852480    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mvqp\" (UniqueName: \"kubernetes.io/projected/4196c457-6d8d-4a01-b299-653d0b4e4445-kube-api-access-9mvqp\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.852626 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852604    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-socket-dir-parent\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.852704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852641    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f6b6450-cd73-4506-85d1-94ef030fe583-iptables-alerter-script\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:22:57.852754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852703    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.852754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852714    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.852754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852720    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:57.852754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852747    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.852946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852794    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.852946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852821    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-dbus\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:57.852946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852860    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f6b6450-cd73-4506-85d1-94ef030fe583-iptables-alerter-script\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:22:57.852946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852880    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.852946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852916    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-ovn\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852944    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852963    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852972    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-client-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.852998    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlvlb\" (UniqueName: \"kubernetes.io/projected/dfd15a52-52ae-4718-a562-ba25778c39ea-kube-api-access-xlvlb\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853034    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853064    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-netd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853088    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4196c457-6d8d-4a01-b299-653d0b4e4445-audit-log\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853115    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh6vv\" (UniqueName: \"kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853128    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.853171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853149    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853179    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9e91ab02-2291-4937-b729-0422ee58256b-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853207    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4196c457-6d8d-4a01-b299-653d0b4e4445-audit-log\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853215    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853198    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853265    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853283    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-system-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853306    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853362    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-bin\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853389    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmrdn\" (UniqueName: \"kubernetes.io/projected/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kube-api-access-qmrdn\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853415    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853442    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853511    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853543    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853551    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.853630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853610    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-kubelet\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853649    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853686    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853767    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9e91ab02-2291-4937-b729-0422ee58256b-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853780    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysconfig\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853806    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853813    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853861    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853883    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-env-overrides\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853899    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-modprobe-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853903    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853915    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-kubernetes\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853946    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-cni-binary-copy\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853974    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d457444-8760-482f-a08d-7b20024db9fa-tmp-dir\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.853999    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhw5z\" (UniqueName: \"kubernetes.io/projected/01fe56b3-4ca9-416a-8480-e199577ac2d5-kube-api-access-nhw5z\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854026    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn7q4\" (UniqueName: \"kubernetes.io/projected/8308da23-cdc6-4e50-91a2-9b158e17ca3b-kube-api-access-cn7q4\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854054    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-kubelet\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854066    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854082    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgckl\" (UniqueName: \"kubernetes.io/projected/ad0e377c-18f5-45ee-aabf-8b228f379db0-kube-api-access-wgckl\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854087    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d457444-8760-482f-a08d-7b20024db9fa-tmp-dir\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854126    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-systemd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854150    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-run\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854148    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-env-overrides\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854180    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854191    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-metrics-client-ca\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854202    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-cni-binary-copy\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854214    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cnibin\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854239    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-log-socket\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854269    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854296    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-wtmp\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.854306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854328    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-metrics-client-ca\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854334    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854485    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-grpc-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854512    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz2b8\" (UniqueName: \"kubernetes.io/projected/8afabbf8-4122-42aa-95c6-75e6d41bf40d-kube-api-access-mz2b8\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854532    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-multus-certs\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854548    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854553    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-etc-kubernetes\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854593    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fd5c8298-1f37-4c24-bb52-1536b0e93028-original-pull-secret\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854598    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854703    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zsf4\" (UniqueName: \"kubernetes.io/projected/9e91ab02-2291-4937-b729-0422ee58256b-kube-api-access-8zsf4\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854711    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-grpc-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854741    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-systemd-units\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854772    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854831    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854896    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-metrics-server-audit-profiles\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.854987    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855011    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-conf\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855105    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-tmp\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855143    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-kubelet-config\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855190    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-tmp\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855193    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-netns\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855214    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4196c457-6d8d-4a01-b299-653d0b4e4445-metrics-server-audit-profiles\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855231    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-node-log\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855262    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-script-lib\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855295    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92z8d\" (UniqueName: \"kubernetes.io/projected/3ead9dc1-5785-432c-a28d-6dd002c4dc27-kube-api-access-92z8d\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855372    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-tuned\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855416    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-etc-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855448    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855473    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-script-lib\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855475    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-tuned\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855474    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-socket-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855515    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.855522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855537    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ce2518fb-0bf8-4628-9927-c46800cb2b3c-serviceca\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855579    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ead9dc1-5785-432c-a28d-6dd002c4dc27-metrics-client-ca\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855592    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855660    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-multus\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855715    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855762    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-config\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855770    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ead9dc1-5785-432c-a28d-6dd002c4dc27-metrics-client-ca\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855798    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-client-certs\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855849    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-var-lib-kubelet\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855887    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855895    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovnkube-config\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855900    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj9zk\" (UniqueName: \"kubernetes.io/projected/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-kube-api-access-fj9zk\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855956    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-sys\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855963    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-client-certs\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.855988    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s49km\" (UniqueName: \"kubernetes.io/projected/6d457444-8760-482f-a08d-7b20024db9fa-kube-api-access-s49km\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856028    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856045    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856061    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-var-lib-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856086    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-tls\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856111    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485ddda6-f359-4513-9a89-239467e9f03a-cert\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856138    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-accelerators-collector-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856142    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ce2518fb-0bf8-4628-9927-c46800cb2b3c-serviceca\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856164    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-device-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856239    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856630    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-secret-metrics-server-tls\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856719    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.856744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856726    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856769    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856786    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-conf-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856788    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-accelerators-collector-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856823    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856860    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/485ddda6-f359-4513-9a89-239467e9f03a-cert\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856859    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.856991    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2518fb-0bf8-4628-9927-c46800cb2b3c-host\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857113    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857144    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857062    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857234    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857274    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-sys-fs\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857307    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-systemd\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857334    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b0deac8-c535-4d85-b27e-968b22b55c14-konnectivity-ca\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857365    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-config-volume\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857397    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9zhc\" (UniqueName: \"kubernetes.io/projected/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-kube-api-access-j9zhc\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857429    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857462    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857479    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fagdf02i9kd0c\""
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857490    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-textfile\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857523    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857556    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857658    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857720    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857590    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857784    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-k8s-cni-cncf-io\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857818    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f6b6450-cd73-4506-85d1-94ef030fe583-host-slash\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:22:57.857843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857848    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-root\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857882    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mtgdn\""
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857889    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858043    5647 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:57.858090    5647 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858197    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.857881    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z6th\" (UniqueName: \"kubernetes.io/projected/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-api-access-9z6th\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858376    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-lib-modules\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858416    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-daemon-config\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858450    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858486    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-metrics-tls\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858517    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858543    5647 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858548    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/395f30f1-f5f2-4164-90bd-02bfbaca7d31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858565    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gnjcg\""
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:22:57.858575    5647 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-187.us-west-1.compute.internal"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858584    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ead9dc1-5785-432c-a28d-6dd002c4dc27-tmp\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858639    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-host\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858688    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-cnibin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858692    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ead9dc1-5785-432c-a28d-6dd002c4dc27-tmp\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858726    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77hn5\" (UniqueName: \"kubernetes.io/projected/ce2518fb-0bf8-4628-9927-c46800cb2b3c-kube-api-access-77hn5\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:22:57.859052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.858768    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859338    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859413    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/395f30f1-f5f2-4164-90bd-02bfbaca7d31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859479    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-slash\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859514    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859520    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859544    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-netns\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859610    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/395f30f1-f5f2-4164-90bd-02bfbaca7d31-crio-socket\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859625    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0e377c-18f5-45ee-aabf-8b228f379db0-metrics-certs\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859647    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01fe56b3-4ca9-416a-8480-e199577ac2d5-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859694    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b0deac8-c535-4d85-b27e-968b22b55c14-konnectivity-ca\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859699    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-bin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859719    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-textfile\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859735    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsjsb\" (UniqueName: \"kubernetes.io/projected/1f6b6450-cd73-4506-85d1-94ef030fe583-kube-api-access-zsjsb\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859767    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckdgq\" (UniqueName: \"kubernetes.io/projected/5db69683-41ca-4f88-a7ab-5a1174170725-kube-api-access-ckdgq\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859793    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f768m\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859849    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovn-node-metrics-cert\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859900    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-daemon-config\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.859920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.859900    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-metrics-tls\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860042    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-hostroot\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860077    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkhzs\" (UniqueName: \"kubernetes.io/projected/485ddda6-f359-4513-9a89-239467e9f03a-kube-api-access-vkhzs\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860110    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d457444-8760-482f-a08d-7b20024db9fa-hosts-file\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860142    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860174    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-tmp-dir\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860207    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860235    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860267    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-system-cni-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860302    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-sys\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860332    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860349    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9e91ab02-2291-4937-b729-0422ee58256b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860365    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-tls\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.860681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860609    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01fe56b3-4ca9-416a-8480-e199577ac2d5-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:57.861259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860797    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-config-volume\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:57.861259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.860983    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.861259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861049    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.861259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861100    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:57.861259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861132    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-registration-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.861259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861167    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b0deac8-c535-4d85-b27e-968b22b55c14-agent-certs\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:22:57.861259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861198    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.861259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861232    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/395f30f1-f5f2-4164-90bd-02bfbaca7d31-data-volume\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.861604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861357    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/395f30f1-f5f2-4164-90bd-02bfbaca7d31-data-volume\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.861604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861430    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-tmp-dir\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:57.861604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861592    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-tls\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.861882 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861861    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.862086 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861994    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8308da23-cdc6-4e50-91a2-9b158e17ca3b-ovn-node-metrics-cert\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.862086 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.862043    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b0deac8-c535-4d85-b27e-968b22b55c14-agent-certs\") pod \"konnectivity-agent-qj6mk\" (UID: \"0b0deac8-c535-4d85-b27e-968b22b55c14\") " pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:22:57.862321 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.861853    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.862373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.862329    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.862703 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.862657    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01fe56b3-4ca9-416a-8480-e199577ac2d5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:57.863395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.863370    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4196c457-6d8d-4a01-b299-653d0b4e4445-client-ca-bundle\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.863519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.863499    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-tls\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.864100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.864083    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.866968 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.866954    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3ead9dc1-5785-432c-a28d-6dd002c4dc27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.912840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.912714    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlvlb\" (UniqueName: \"kubernetes.io/projected/dfd15a52-52ae-4718-a562-ba25778c39ea-kube-api-access-xlvlb\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.913000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.912937    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmrdn\" (UniqueName: \"kubernetes.io/projected/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kube-api-access-qmrdn\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.913059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.913025    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mvqp\" (UniqueName: \"kubernetes.io/projected/4196c457-6d8d-4a01-b299-653d0b4e4445-kube-api-access-9mvqp\") pod \"metrics-server-6ddc555bd7-sljjr\" (UID: \"4196c457-6d8d-4a01-b299-653d0b4e4445\") " pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:22:57.913112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.913061    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh6vv\" (UniqueName: \"kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv\") pod \"console-59c56bd6bd-82w6k\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") " pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:22:57.913112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.913063    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgckl\" (UniqueName: \"kubernetes.io/projected/ad0e377c-18f5-45ee-aabf-8b228f379db0-kube-api-access-wgckl\") pod \"network-metrics-daemon-vr2lh\" (UID: \"ad0e377c-18f5-45ee-aabf-8b228f379db0\") " pod="openshift-multus/network-metrics-daemon-vr2lh"
Nov 03 22:22:57.913221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.913179    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhw5z\" (UniqueName: \"kubernetes.io/projected/01fe56b3-4ca9-416a-8480-e199577ac2d5-kube-api-access-nhw5z\") pod \"openshift-state-metrics-7bd7489957-gjlm7\" (UID: \"01fe56b3-4ca9-416a-8480-e199577ac2d5\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-gjlm7"
Nov 03 22:22:57.915159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.915137    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn7q4\" (UniqueName: \"kubernetes.io/projected/8308da23-cdc6-4e50-91a2-9b158e17ca3b-kube-api-access-cn7q4\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.915496 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.915476    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj9zk\" (UniqueName: \"kubernetes.io/projected/fe23ba2a-adba-4d25-b08c-cbcd0d0c0565-kube-api-access-fj9zk\") pod \"dns-default-9fnc7\" (UID: \"fe23ba2a-adba-4d25-b08c-cbcd0d0c0565\") " pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:57.915541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.915512    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zsf4\" (UniqueName: \"kubernetes.io/projected/9e91ab02-2291-4937-b729-0422ee58256b-kube-api-access-8zsf4\") pod \"kube-state-metrics-5dc8d7d89b-tslkb\" (UID: \"9e91ab02-2291-4937-b729-0422ee58256b\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-tslkb"
Nov 03 22:22:57.916335 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.916311    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s49km\" (UniqueName: \"kubernetes.io/projected/6d457444-8760-482f-a08d-7b20024db9fa-kube-api-access-s49km\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:22:57.916584 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.916559    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz2b8\" (UniqueName: \"kubernetes.io/projected/8afabbf8-4122-42aa-95c6-75e6d41bf40d-kube-api-access-mz2b8\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.916827 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.916809    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92z8d\" (UniqueName: \"kubernetes.io/projected/3ead9dc1-5785-432c-a28d-6dd002c4dc27-kube-api-access-92z8d\") pod \"thanos-querier-b46bc6cb-p4dg9\" (UID: \"3ead9dc1-5785-432c-a28d-6dd002c4dc27\") " pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:22:57.920628 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.920596    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsjsb\" (UniqueName: \"kubernetes.io/projected/1f6b6450-cd73-4506-85d1-94ef030fe583-kube-api-access-zsjsb\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:22:57.921052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.921035    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jxsw\" (UniqueName: \"kubernetes.io/projected/7ac23160-d159-4e38-9a63-691052ae8211-kube-api-access-4jxsw\") pod \"network-check-target-g9t4g\" (UID: \"7ac23160-d159-4e38-9a63-691052ae8211\") " pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:22:57.921435 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.921414    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77hn5\" (UniqueName: \"kubernetes.io/projected/ce2518fb-0bf8-4628-9927-c46800cb2b3c-kube-api-access-77hn5\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:22:57.921718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.921698    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9zhc\" (UniqueName: \"kubernetes.io/projected/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-kube-api-access-j9zhc\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.921835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.921816    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckdgq\" (UniqueName: \"kubernetes.io/projected/5db69683-41ca-4f88-a7ab-5a1174170725-kube-api-access-ckdgq\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.921924 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.921911    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkhzs\" (UniqueName: \"kubernetes.io/projected/485ddda6-f359-4513-9a89-239467e9f03a-kube-api-access-vkhzs\") pod \"ingress-canary-z2ljt\" (UID: \"485ddda6-f359-4513-9a89-239467e9f03a\") " pod="openshift-ingress-canary/ingress-canary-z2ljt"
Nov 03 22:22:57.925202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.925183    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f768m\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m\") pod \"alertmanager-main-0\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:22:57.928436 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.928420    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z6th\" (UniqueName: \"kubernetes.io/projected/395f30f1-f5f2-4164-90bd-02bfbaca7d31-kube-api-access-9z6th\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.961543 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961509    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-kubelet-config\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:57.961543 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961544    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-netns\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961566    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-node-log\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961592    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-etc-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961615    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961638    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-socket-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961638    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-netns\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961657    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-node-log\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961638    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-kubelet-config\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961722    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-multus\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961661    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-etc-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961715    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961717    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-socket-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.961764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961691    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-multus\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961780    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-var-lib-kubelet\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961814    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-sys\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961831    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-var-lib-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961851    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-device-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961886    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-sys\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961893    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-var-lib-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961900    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-device-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961924    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-conf-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961927    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-var-lib-kubelet\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961946    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961953    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-conf-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961963    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2518fb-0bf8-4628-9927-c46800cb2b3c-host\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961990    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2518fb-0bf8-4628-9927-c46800cb2b3c-host\") pod \"node-ca-84hzv\" (UID: \"ce2518fb-0bf8-4628-9927-c46800cb2b3c\") " pod="openshift-image-registry/node-ca-84hzv"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.961994    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-sys-fs\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962000    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962011    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-systemd\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962024    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-sys-fs\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962032    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962050    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-k8s-cni-cncf-io\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962064    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f6b6450-cd73-4506-85d1-94ef030fe583-host-slash\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962076    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-root\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962111    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f6b6450-cd73-4506-85d1-94ef030fe583-host-slash\") pod \"iptables-alerter-lqjh7\" (UID: \"1f6b6450-cd73-4506-85d1-94ef030fe583\") " pod="openshift-network-operator/iptables-alerter-lqjh7"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962107    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-root\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962133    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-lib-modules\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962135    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962150    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-host\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962171    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-cnibin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962177    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-k8s-cni-cncf-io\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962193    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-slash\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962216    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-host\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962194    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-systemd\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962226    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-slash\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962218    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962227    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-cnibin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962262    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-netns\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962279    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/395f30f1-f5f2-4164-90bd-02bfbaca7d31-crio-socket\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962278    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-lib-modules\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962295    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-netns\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962287    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.962282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962306    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-bin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962318    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/395f30f1-f5f2-4164-90bd-02bfbaca7d31-crio-socket\") pod \"insights-runtime-extractor-lmz29\" (UID: \"395f30f1-f5f2-4164-90bd-02bfbaca7d31\") " pod="openshift-insights/insights-runtime-extractor-lmz29"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962343    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-hostroot\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962361    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d457444-8760-482f-a08d-7b20024db9fa-hosts-file\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962389    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-cni-bin\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962402    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-hostroot\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962407    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962445    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-system-cni-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962407    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d457444-8760-482f-a08d-7b20024db9fa-hosts-file\") pod \"node-resolver-pvgnl\" (UID: \"6d457444-8760-482f-a08d-7b20024db9fa\") " pod="openshift-dns/node-resolver-pvgnl"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962469    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-sys\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962500    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-system-cni-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962504    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962501    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962511    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-sys\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962542    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-registration-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962557    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962561    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-socket-dir-parent\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962584    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8-registration-dir\") pod \"aws-ebs-csi-driver-node-p8q29\" (UID: \"dc6bc3da-6c5f-4bb6-9369-6967db1ad8a8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p8q29"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962592    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-multus-socket-dir-parent\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962595    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962620    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962644    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-dbus\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962699    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-openvswitch\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962710    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962715    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-ovn\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962730    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fd5c8298-1f37-4c24-bb52-1536b0e93028-dbus\") pod \"global-pull-secret-syncer-zhsp8\" (UID: \"fd5c8298-1f37-4c24-bb52-1536b0e93028\") " pod="kube-system/global-pull-secret-syncer-zhsp8"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962746    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-ovn\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962746    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-netd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962767    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-netd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962797    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-system-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962821    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-bin\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962849    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-cni-bin\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962849    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-kubelet\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962883    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-host-kubelet\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962889    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysconfig\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962894    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-system-cni-dir\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962915    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysconfig\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962917    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-modprobe-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962957    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-kubernetes\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.962986    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-kubelet\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963010    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-kubernetes\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963014    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-modprobe-d\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963010    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-systemd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963038    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-run\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963053    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-var-lib-kubelet\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963055    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cnibin\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963079    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-log-socket\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963091    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-run-systemd\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963095    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-wtmp\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963111    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-run\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963119    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db69683-41ca-4f88-a7ab-5a1174170725-node-exporter-wtmp\") pod \"node-exporter-8rxz5\" (UID: \"5db69683-41ca-4f88-a7ab-5a1174170725\") " pod="openshift-monitoring/node-exporter-8rxz5"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963132    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-multus-certs\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963140    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9f62230-4ed4-49ec-bc29-a69f0b46bd8f-cnibin\") pod \"multus-additional-cni-plugins-8ck4c\" (UID: \"b9f62230-4ed4-49ec-bc29-a69f0b46bd8f\") " pod="openshift-multus/multus-additional-cni-plugins-8ck4c"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963159    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-log-socket\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963162    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-host-run-multus-certs\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963167    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-etc-kubernetes\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963197    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-systemd-units\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963197    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dfd15a52-52ae-4718-a562-ba25778c39ea-etc-kubernetes\") pod \"multus-rkgdk\" (UID: \"dfd15a52-52ae-4718-a562-ba25778c39ea\") " pod="openshift-multus/multus-rkgdk"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963220    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8308da23-cdc6-4e50-91a2-9b158e17ca3b-systemd-units\") pod \"ovnkube-node-d7l9z\" (UID: \"8308da23-cdc6-4e50-91a2-9b158e17ca3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963223    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-conf\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:57.963784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:57.963299    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8afabbf8-4122-42aa-95c6-75e6d41bf40d-etc-sysctl-conf\") pod \"tuned-2lfgw\" (UID: \"8afabbf8-4122-42aa-95c6-75e6d41bf40d\") " pod="openshift-cluster-node-tuning-operator/tuned-2lfgw"
Nov 03 22:22:58.051797 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:58.051779    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:22:58.337322 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:58.337294    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9fnc7"
Nov 03 22:22:59.739080 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:59.739045    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qj6mk"
Nov 03 22:22:59.786087 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:22:59.786051    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c56bd6bd-82w6k" event={"ID":"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3","Type":"ContainerStarted","Data":"ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269"}
Nov 03 22:23:00.437218 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:00.437183    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:23:01.606324 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:01.606294    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9"
Nov 03 22:23:02.752253 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:02.752222    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:23:02.816478 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:02.816447    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:23:03.390545 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.390511    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-f8cf5585d-7jxcg"]
Nov 03 22:23:03.394041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.394022    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.501589 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.501555    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-oauth-serving-cert\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.501802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.501613    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-trusted-ca-bundle\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.501802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.501702    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk4k5\" (UniqueName: \"kubernetes.io/projected/0929767e-35b2-4f25-86b1-0d5b197be321-kube-api-access-xk4k5\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.501802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.501776    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-console-config\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.501942 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.501811    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-oauth-config\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.501980 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.501940    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-serving-cert\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.502033 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.501979    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0929767e-35b2-4f25-86b1-0d5b197be321-tmp\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.502033 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.502015    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-service-ca\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.510734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.510706    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:23:03.510956 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.510942    5647 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 22:23:03.595014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.594978    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 03 22:23:03.595597 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.595579    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f8cf5585d-7jxcg"]
Nov 03 22:23:03.603462 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.603433    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-trusted-ca-bundle\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.603541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.603472    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk4k5\" (UniqueName: \"kubernetes.io/projected/0929767e-35b2-4f25-86b1-0d5b197be321-kube-api-access-xk4k5\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.603541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.603497    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-console-config\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.603648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.603543    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-oauth-config\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.603744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.603659    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-serving-cert\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.603744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.603716    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0929767e-35b2-4f25-86b1-0d5b197be321-tmp\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.603838 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.603747    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-service-ca\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.603838 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.603790    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-oauth-serving-cert\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.604071 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.604039    5647 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 22:23:03.604425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.604242    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0929767e-35b2-4f25-86b1-0d5b197be321-tmp\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.604425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.604378    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-console-config\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.604767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.604594    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-service-ca\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.604767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.604690    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-trusted-ca-bundle\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.604767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.604736    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-oauth-serving-cert\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.606389 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.606369    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-oauth-config\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.606559 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.606540    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-serving-cert\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.664359 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.664295    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk4k5\" (UniqueName: \"kubernetes.io/projected/0929767e-35b2-4f25-86b1-0d5b197be321-kube-api-access-xk4k5\") pod \"console-f8cf5585d-7jxcg\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") " pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:03.704750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:03.704713    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:04.077220 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:04.077189    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f8cf5585d-7jxcg"]
Nov 03 22:23:04.080203 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:23:04.080172    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0929767e_35b2_4f25_86b1_0d5b197be321.slice/crio-8bc518cb0100d6ab75154be638daa2c1f87d2d253661f979e6c730466f83edec WatchSource:0}: Error finding container 8bc518cb0100d6ab75154be638daa2c1f87d2d253661f979e6c730466f83edec: Status 404 returned error can't find the container with id 8bc518cb0100d6ab75154be638daa2c1f87d2d253661f979e6c730466f83edec
Nov 03 22:23:04.486175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:04.486145    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:23:04.491106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:04.491079    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:23:04.806317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:04.806221    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8cf5585d-7jxcg" event={"ID":"0929767e-35b2-4f25-86b1-0d5b197be321","Type":"ContainerStarted","Data":"4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0"}
Nov 03 22:23:04.806317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:04.806276    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8cf5585d-7jxcg" event={"ID":"0929767e-35b2-4f25-86b1-0d5b197be321","Type":"ContainerStarted","Data":"8bc518cb0100d6ab75154be638daa2c1f87d2d253661f979e6c730466f83edec"}
Nov 03 22:23:04.809956 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:04.809931    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:23:04.923253 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:04.923092    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-f8cf5585d-7jxcg" podStartSLOduration=1.923075291 podStartE2EDuration="1.923075291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:04.923053968 +0000 UTC m=+8.363591228" watchObservedRunningTime="2026-11-03 22:23:04.923075291 +0000 UTC m=+8.363612550"
Nov 03 22:23:05.544413 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:05.544386    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7l9z"
Nov 03 22:23:12.299879 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:12.299795    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:12.791759 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:12.791730    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6ddc555bd7-sljjr"
Nov 03 22:23:13.688331 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:13.688294    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:13.692974 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:13.692949    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:13.842865 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:13.842829    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:23:13.990987 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:13.990950    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c56bd6bd-82w6k"]
Nov 03 22:23:16.043542 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:23:16.043506    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:23:22.754977 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.754948    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:23:22.755380 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.755237    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="alertmanager" containerID="cri-o://97282f71b3cba780ab4aa230ad6714a6e2b9d9aeef9ac63697becf1c34bfeac1" gracePeriod=120
Nov 03 22:23:22.755380 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.755301    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy-metric" containerID="cri-o://ff97d24c7d876bebbbb7fa344ad849586f1b7aafb1b6b2c0c2d39b5ea5c29f24" gracePeriod=120
Nov 03 22:23:22.755507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.755367    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy" containerID="cri-o://6f2aff40ecbe743b52300cb09538554c1aef616b9c9c34407c2874075f911d3b" gracePeriod=120
Nov 03 22:23:22.755507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.755323    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy-web" containerID="cri-o://7225ca7443734aaffc19b8897de44628fd1036915289bfbb09c67cf5a72fe50f" gracePeriod=120
Nov 03 22:23:22.755507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.755395    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="prom-label-proxy" containerID="cri-o://e07c32aa30f619b5c5bb7d57bdb0d82ff3898f6750c0b3d74a66c409f21dedca" gracePeriod=120
Nov 03 22:23:22.755507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.755363    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="config-reloader" containerID="cri-o://4aeb105f7e82e22064df94e06995a158c27c9ab5aa6f5fa009e03842578e61e4" gracePeriod=120
Nov 03 22:23:22.877353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877328    5647 generic.go:356] "Generic (PLEG): container finished" podID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerID="e07c32aa30f619b5c5bb7d57bdb0d82ff3898f6750c0b3d74a66c409f21dedca" exitCode=0
Nov 03 22:23:22.877353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877351    5647 generic.go:356] "Generic (PLEG): container finished" podID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerID="ff97d24c7d876bebbbb7fa344ad849586f1b7aafb1b6b2c0c2d39b5ea5c29f24" exitCode=0
Nov 03 22:23:22.877484 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877357    5647 generic.go:356] "Generic (PLEG): container finished" podID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerID="6f2aff40ecbe743b52300cb09538554c1aef616b9c9c34407c2874075f911d3b" exitCode=0
Nov 03 22:23:22.877484 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877365    5647 generic.go:356] "Generic (PLEG): container finished" podID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerID="4aeb105f7e82e22064df94e06995a158c27c9ab5aa6f5fa009e03842578e61e4" exitCode=0
Nov 03 22:23:22.877484 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877374    5647 generic.go:356] "Generic (PLEG): container finished" podID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerID="97282f71b3cba780ab4aa230ad6714a6e2b9d9aeef9ac63697becf1c34bfeac1" exitCode=0
Nov 03 22:23:22.877484 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877394    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerDied","Data":"e07c32aa30f619b5c5bb7d57bdb0d82ff3898f6750c0b3d74a66c409f21dedca"}
Nov 03 22:23:22.877484 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877459    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerDied","Data":"ff97d24c7d876bebbbb7fa344ad849586f1b7aafb1b6b2c0c2d39b5ea5c29f24"}
Nov 03 22:23:22.877484 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877486    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerDied","Data":"6f2aff40ecbe743b52300cb09538554c1aef616b9c9c34407c2874075f911d3b"}
Nov 03 22:23:22.877656 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877511    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerDied","Data":"4aeb105f7e82e22064df94e06995a158c27c9ab5aa6f5fa009e03842578e61e4"}
Nov 03 22:23:22.877656 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:22.877534    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerDied","Data":"97282f71b3cba780ab4aa230ad6714a6e2b9d9aeef9ac63697becf1c34bfeac1"}
Nov 03 22:23:23.884659 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:23.884629    5647 generic.go:356] "Generic (PLEG): container finished" podID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerID="7225ca7443734aaffc19b8897de44628fd1036915289bfbb09c67cf5a72fe50f" exitCode=0
Nov 03 22:23:23.885003 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:23.884680    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerDied","Data":"7225ca7443734aaffc19b8897de44628fd1036915289bfbb09c67cf5a72fe50f"}
Nov 03 22:23:24.002959 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.002935    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:24.079303 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079207    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079303 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079257    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079303 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079296    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079325    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079354    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079397    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079425    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079461    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079486    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079538    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079569    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.079602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079597    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:23:24.080138 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079602    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.080138 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.079749    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy\") pod \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\" (UID: \"7ea0760a-ec14-4dce-b11b-d746d72de9ee\") "
Nov 03 22:23:24.080138 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.080057    5647 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-main-db\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.083585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.083526    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:24.084107 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.083857    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:23:24.084107 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.083933    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:23:24.084107 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.084058    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:24.084301 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.084114    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:23:24.084301 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.084268    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:23:24.084301 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.084272    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:24.084453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.084404    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:24.084534 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.084510    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:24.084623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.084590    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "kube-api-access-f768m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:23:24.087916 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.087893    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:24.095061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.095035    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config" pod "7ea0760a-ec14-4dce-b11b-d746d72de9ee" (UID: "7ea0760a-ec14-4dce-b11b-d746d72de9ee"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:24.181440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181401    5647 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-tls-assets\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181432    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-metrics-client-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181445    5647 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181455    5647 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181465    5647 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181473    5647 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181482    5647 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea0760a-ec14-4dce-b11b-d746d72de9ee-config-out\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181508    5647 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-web-config\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181518    5647 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-secret-alertmanager-main-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181527    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f768m\" (UniqueName: \"kubernetes.io/projected/7ea0760a-ec14-4dce-b11b-d746d72de9ee-kube-api-access-f768m\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181536    5647 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea0760a-ec14-4dce-b11b-d746d72de9ee-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.181544    5647 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ea0760a-ec14-4dce-b11b-d746d72de9ee-cluster-tls-config\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:24.891431 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.891399    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ea0760a-ec14-4dce-b11b-d746d72de9ee","Type":"ContainerDied","Data":"a26ae6a44a25759ab38d917b596be6f63ade0ef65561db450fb7468662e5717e"}
Nov 03 22:23:24.891833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.891452    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:24.891833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.891483    5647 scope.go:122] "RemoveContainer" containerID="e07c32aa30f619b5c5bb7d57bdb0d82ff3898f6750c0b3d74a66c409f21dedca"
Nov 03 22:23:24.899108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.899093    5647 scope.go:122] "RemoveContainer" containerID="ff97d24c7d876bebbbb7fa344ad849586f1b7aafb1b6b2c0c2d39b5ea5c29f24"
Nov 03 22:23:24.905885 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.905853    5647 scope.go:122] "RemoveContainer" containerID="6f2aff40ecbe743b52300cb09538554c1aef616b9c9c34407c2874075f911d3b"
Nov 03 22:23:24.912493 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.912475    5647 scope.go:122] "RemoveContainer" containerID="7225ca7443734aaffc19b8897de44628fd1036915289bfbb09c67cf5a72fe50f"
Nov 03 22:23:24.918951 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.918933    5647 scope.go:122] "RemoveContainer" containerID="4aeb105f7e82e22064df94e06995a158c27c9ab5aa6f5fa009e03842578e61e4"
Nov 03 22:23:24.925816 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.925801    5647 scope.go:122] "RemoveContainer" containerID="97282f71b3cba780ab4aa230ad6714a6e2b9d9aeef9ac63697becf1c34bfeac1"
Nov 03 22:23:24.932534 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:24.932506    5647 scope.go:122] "RemoveContainer" containerID="0f11b7165af4bc6c0c544820b2a21aa30ecc72f833193c1e810261b33cc3a926"
Nov 03 22:23:25.106726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.106695    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:23:25.156392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.156350    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:23:25.156798 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.156781    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:23:25.157039 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157027    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="prom-label-proxy"
Nov 03 22:23:25.157077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157041    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="prom-label-proxy"
Nov 03 22:23:25.157077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157052    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="config-reloader"
Nov 03 22:23:25.157077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157058    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="config-reloader"
Nov 03 22:23:25.157077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157069    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="init-config-reloader"
Nov 03 22:23:25.157077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157074    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="init-config-reloader"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157090    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy-metric"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157095    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy-metric"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157105    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="alertmanager"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157113    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="alertmanager"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157127    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy-web"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157133    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy-web"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157144    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157149    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157192    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="init-config-reloader"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157201    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy-metric"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157207    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="alertmanager"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157213    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy-web"
Nov 03 22:23:25.157217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157218    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="kube-rbac-proxy"
Nov 03 22:23:25.157578 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157224    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="prom-label-proxy"
Nov 03 22:23:25.157578 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.157229    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" containerName="config-reloader"
Nov 03 22:23:25.160530 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.160510    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.213203 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.213175    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 03 22:23:25.213395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.213379    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 03 22:23:25.213476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.213379    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 03 22:23:25.213476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.213386    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 03 22:23:25.213476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.213415    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 03 22:23:25.213476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.213425    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 03 22:23:25.262597 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.262562    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 03 22:23:25.262788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.262611    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-84pw6\""
Nov 03 22:23:25.262788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.262616    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 03 22:23:25.262788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.262743    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 03 22:23:25.294752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.294719    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-web-config\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.294905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.294766    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-config-out\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.294905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.294789    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295023 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.294916    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295023 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.294952    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295088 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.295039    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjbxf\" (UniqueName: \"kubernetes.io/projected/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-kube-api-access-zjbxf\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295088 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.295078    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.295100    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.295128    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-config-volume\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.295165    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-tls-assets\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.295231    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295311 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.295249    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.295311 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.295280    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.393899 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.393817    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:23:25.396057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396028    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396070    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-web-config\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396117    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-config-out\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396140    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396166    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396189    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396526 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396507    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjbxf\" (UniqueName: \"kubernetes.io/projected/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-kube-api-access-zjbxf\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396653    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396808    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.396955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.396940    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-config-volume\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.397063 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.397047    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-tls-assets\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.397176 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.397161    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.397276 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.397260    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.397463 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.397437    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.397534 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.397066    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.397752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.397725    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.399794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.399749    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.400102 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.400068    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-config-out\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.400600 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.400577    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-web-config\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.401442 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.401417    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-config-volume\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.401805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.401786    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.401996 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.401956    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.402300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.402279    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.402937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.402913    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.403036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.403020    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-tls-assets\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.457580 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.457545    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjbxf\" (UniqueName: \"kubernetes.io/projected/36f9a8e0-049b-40e4-b79b-cf5b9bf28149-kube-api-access-zjbxf\") pod \"alertmanager-main-0\" (UID: \"36f9a8e0-049b-40e4-b79b-cf5b9bf28149\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.477418 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.477393    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 22:23:25.904525 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:25.904495    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 22:23:25.907770 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:23:25.907741    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f9a8e0_049b_40e4_b79b_cf5b9bf28149.slice/crio-85dd27c16e019a73bfa565857ac0dc6e98dd02b595fb6752b0e61d81152a76f7 WatchSource:0}: Error finding container 85dd27c16e019a73bfa565857ac0dc6e98dd02b595fb6752b0e61d81152a76f7: Status 404 returned error can't find the container with id 85dd27c16e019a73bfa565857ac0dc6e98dd02b595fb6752b0e61d81152a76f7
Nov 03 22:23:26.418662 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.418629    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"]
Nov 03 22:23:26.420660 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.420644    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.472814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.472787    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 03 22:23:26.472814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.472812    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 03 22:23:26.473021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.472887    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 03 22:23:26.473179 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.473162    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 03 22:23:26.473222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.473162    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 03 22:23:26.505830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.505807    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.505915 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.505850    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-telemeter-client-tls\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.505915 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.505870    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.506026 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.506010    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-serving-certs-ca-bundle\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.506061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.506035    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-metrics-client-ca\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.506094 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.506061    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w64v2\" (UniqueName: \"kubernetes.io/projected/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-kube-api-access-w64v2\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.506128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.506109    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-federate-client-tls\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.506210 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.506195    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-secret-telemeter-client\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.522235 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.522210    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zxvpv\""
Nov 03 22:23:26.522996 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.522980    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 03 22:23:26.606850 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.606819    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-telemeter-client-tls\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.606859    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.606890    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-serving-certs-ca-bundle\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.606906    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-metrics-client-ca\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.606921    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w64v2\" (UniqueName: \"kubernetes.io/projected/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-kube-api-access-w64v2\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.606945    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-federate-client-tls\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.606985    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-secret-telemeter-client\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607299 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.607016    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.607968    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.607979    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-serving-certs-ca-bundle\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.607993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.607988    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-metrics-client-ca\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.610128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.610104    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-telemeter-client-tls\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.610256 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.610235    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.610296 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.610257    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-federate-client-tls\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.610327 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.610290    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-secret-telemeter-client\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.643745 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.643718    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"]
Nov 03 22:23:26.667544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.667517    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w64v2\" (UniqueName: \"kubernetes.io/projected/9c408a46-d8c0-4c25-a708-2fbe5f78a1f9-kube-api-access-w64v2\") pod \"telemeter-client-6b759b6f89-dqvbb\" (UID: \"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9\") " pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.694654 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.694591    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ea0760a-ec14-4dce-b11b-d746d72de9ee" path="/var/lib/kubelet/pods/7ea0760a-ec14-4dce-b11b-d746d72de9ee/volumes"
Nov 03 22:23:26.732926 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.732894    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"
Nov 03 22:23:26.900103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.900073    5647 generic.go:356] "Generic (PLEG): container finished" podID="36f9a8e0-049b-40e4-b79b-cf5b9bf28149" containerID="a1bf31088ab58da9632be4893b8f0a118dd3eb388916363eedd865561769075d" exitCode=0
Nov 03 22:23:26.900277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.900140    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"36f9a8e0-049b-40e4-b79b-cf5b9bf28149","Type":"ContainerDied","Data":"a1bf31088ab58da9632be4893b8f0a118dd3eb388916363eedd865561769075d"}
Nov 03 22:23:26.900277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:26.900189    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"36f9a8e0-049b-40e4-b79b-cf5b9bf28149","Type":"ContainerStarted","Data":"85dd27c16e019a73bfa565857ac0dc6e98dd02b595fb6752b0e61d81152a76f7"}
Nov 03 22:23:27.201788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:27.201727    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b759b6f89-dqvbb"]
Nov 03 22:23:27.205030 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:23:27.204993    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c408a46_d8c0_4c25_a708_2fbe5f78a1f9.slice/crio-a5708ae416608d718e7e0786d906e65e70f1683347105da6ab74e9e3679181a9 WatchSource:0}: Error finding container a5708ae416608d718e7e0786d906e65e70f1683347105da6ab74e9e3679181a9: Status 404 returned error can't find the container with id a5708ae416608d718e7e0786d906e65e70f1683347105da6ab74e9e3679181a9
Nov 03 22:23:27.904755 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:27.904717    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb" event={"ID":"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9","Type":"ContainerStarted","Data":"a5708ae416608d718e7e0786d906e65e70f1683347105da6ab74e9e3679181a9"}
Nov 03 22:23:27.908059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:27.908031    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"36f9a8e0-049b-40e4-b79b-cf5b9bf28149","Type":"ContainerStarted","Data":"6f4bf6c786a3f8cc8a0bcd9687e057a0bc6b6832a067292130bcfe6f1bf4677e"}
Nov 03 22:23:27.908155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:27.908088    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"36f9a8e0-049b-40e4-b79b-cf5b9bf28149","Type":"ContainerStarted","Data":"ec7a601996946b2ece4514bd46492d3f31113b1a20fe09fea513d389b700f301"}
Nov 03 22:23:27.908155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:27.908113    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"36f9a8e0-049b-40e4-b79b-cf5b9bf28149","Type":"ContainerStarted","Data":"8cb5308db51375b2c9d5ddf846273f167e305dc3497ca27d76fd2e01bcf93565"}
Nov 03 22:23:27.908155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:27.908140    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"36f9a8e0-049b-40e4-b79b-cf5b9bf28149","Type":"ContainerStarted","Data":"5053a84006e8f22991e201900490534921afc4b95d0ed3631240661c143b8f48"}
Nov 03 22:23:27.908260 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:27.908164    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"36f9a8e0-049b-40e4-b79b-cf5b9bf28149","Type":"ContainerStarted","Data":"5df1fb06fda72688c6e7e7cd7e689ebea25356aae9c8470b618b23447ec671b8"}
Nov 03 22:23:27.908260 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:27.908186    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"36f9a8e0-049b-40e4-b79b-cf5b9bf28149","Type":"ContainerStarted","Data":"bdf588de4c67ffd3820de786293caab8471747f80690af4c9ee27849089260c1"}
Nov 03 22:23:28.036210 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:28.036150    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.023987447 podStartE2EDuration="3.036134725s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.147278ms" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:28.035757453 +0000 UTC m=+31.476294725" watchObservedRunningTime="2026-11-03 22:23:28.036134725 +0000 UTC m=+31.476671987"
Nov 03 22:23:29.918610 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:29.918568    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb" event={"ID":"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9","Type":"ContainerStarted","Data":"22dbdecd3fae8089e003062a1f92485bfbcf8ca115d297e957447b3fc18bca81"}
Nov 03 22:23:29.919009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:29.918638    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb" event={"ID":"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9","Type":"ContainerStarted","Data":"242de679144100ae8714dafdcfbfe1560874da2556b772bde7c4b60e96fed1d4"}
Nov 03 22:23:29.919009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:29.918696    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb" event={"ID":"9c408a46-d8c0-4c25-a708-2fbe5f78a1f9","Type":"ContainerStarted","Data":"1939afe05738231a77ca1f030e42670937adaac5e7ae76f0646bcd0193c63476"}
Nov 03 22:23:30.140053 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.139972    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6b759b6f89-dqvbb" podStartSLOduration=2.25802709 podStartE2EDuration="4.139955492s" totalImagesPullingTime="1.881928402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:30.089810252 +0000 UTC m=+33.530347511" watchObservedRunningTime="2026-11-03 22:23:30.139955492 +0000 UTC m=+33.580492750"
Nov 03 22:23:30.528794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.527948    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d65f9c6bb-2kr92"]
Nov 03 22:23:30.531504 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.531474    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.589651 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.589614    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d65f9c6bb-2kr92"]
Nov 03 22:23:30.643010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.642979    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc7wx\" (UniqueName: \"kubernetes.io/projected/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-kube-api-access-dc7wx\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.643171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.643025    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-config\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.643171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.643102    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-tmp\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.643171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.643131    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-service-ca\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.643171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.643156    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-oauth-config\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.643407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.643238    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-serving-cert\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.643407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.643270    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-trusted-ca-bundle\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.643407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.643297    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-oauth-serving-cert\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.744283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.744242    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-serving-cert\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.744476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.744287    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-trusted-ca-bundle\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.744476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.744315    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-oauth-serving-cert\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.744476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.744343    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc7wx\" (UniqueName: \"kubernetes.io/projected/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-kube-api-access-dc7wx\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.744643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.744520    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-config\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.744643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.744582    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-tmp\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.744643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.744618    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-service-ca\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.744826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.744654    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-oauth-config\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.745132 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.745102    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-tmp\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.745265 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.745235    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-oauth-serving-cert\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.745265 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.745245    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-config\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.745438 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.745315    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-service-ca\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.745438 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.745423    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-trusted-ca-bundle\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.746870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.746849    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-serving-cert\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.747474 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.747458    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-oauth-config\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.805052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.804993    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc7wx\" (UniqueName: \"kubernetes.io/projected/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-kube-api-access-dc7wx\") pod \"console-7d65f9c6bb-2kr92\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") " pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:30.841806 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:30.841765    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:31.215721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:31.215687    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d65f9c6bb-2kr92"]
Nov 03 22:23:31.246516 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:23:31.246485    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2bbf368_f12f_4683_bfba_a0ccae8b4a80.slice/crio-59335967588e7c9679a0b98664178f47a19229a71d74b3448686142dea3d356b WatchSource:0}: Error finding container 59335967588e7c9679a0b98664178f47a19229a71d74b3448686142dea3d356b: Status 404 returned error can't find the container with id 59335967588e7c9679a0b98664178f47a19229a71d74b3448686142dea3d356b
Nov 03 22:23:31.929361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:31.929322    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d65f9c6bb-2kr92" event={"ID":"f2bbf368-f12f-4683-bfba-a0ccae8b4a80","Type":"ContainerStarted","Data":"e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e"}
Nov 03 22:23:31.929532 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:31.929379    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d65f9c6bb-2kr92" event={"ID":"f2bbf368-f12f-4683-bfba-a0ccae8b4a80","Type":"ContainerStarted","Data":"59335967588e7c9679a0b98664178f47a19229a71d74b3448686142dea3d356b"}
Nov 03 22:23:32.046847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:32.046788    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7d65f9c6bb-2kr92" podStartSLOduration=2.046770315 podStartE2EDuration="2.046770315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:23:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:23:32.046147045 +0000 UTC m=+35.486684307" watchObservedRunningTime="2026-11-03 22:23:32.046770315 +0000 UTC m=+35.487307573"
Nov 03 22:23:32.819277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:32.819251    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g9t4g"
Nov 03 22:23:39.012211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.012143    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-59c56bd6bd-82w6k" podUID="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" containerName="console" containerID="cri-o://ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269" gracePeriod=15
Nov 03 22:23:39.255801 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.255776    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:23:39.323978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.323909    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv\" (UniqueName: \"kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv\") pod \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") "
Nov 03 22:23:39.323978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.323960    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert\") pod \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") "
Nov 03 22:23:39.324127 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.323992    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config\") pod \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") "
Nov 03 22:23:39.324193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324175    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca\") pod \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") "
Nov 03 22:23:39.324239 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324223    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp\" (UniqueName: \"kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp\") pod \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") "
Nov 03 22:23:39.324295 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324260    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert\") pod \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") "
Nov 03 22:23:39.324350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324308    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config\") pod \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\" (UID: \"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3\") "
Nov 03 22:23:39.324541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324481    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp" pod "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" (UID: "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:23:39.324704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324566    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca" pod "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" (UID: "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:23:39.324789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324716    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert" pod "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" (UID: "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:23:39.324789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324759    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config" pod "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" (UID: "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:23:39.324877 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324802    5647 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:39.324877 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.324817    5647 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-tmp\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:39.326497 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.326475    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config" pod "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" (UID: "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:39.326732 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.326709    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv" pod "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" (UID: "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3"). InnerVolumeSpecName "kube-api-access-zh6vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:23:39.326732 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.326717    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert" pod "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" (UID: "789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:23:39.425777 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.425739    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh6vv\" (UniqueName: \"kubernetes.io/projected/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-kube-api-access-zh6vv\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:39.425777 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.425767    5647 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-serving-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:39.425777 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.425777    5647 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-oauth-config\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:39.425777 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.425786    5647 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-oauth-serving-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:39.426045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.425795    5647 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3-console-config\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:23:39.961954 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.961921    5647 generic.go:356] "Generic (PLEG): container finished" podID="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" containerID="ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269" exitCode=0
Nov 03 22:23:39.962121 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.961971    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c56bd6bd-82w6k"
Nov 03 22:23:39.962121 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.961984    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c56bd6bd-82w6k" event={"ID":"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3","Type":"ContainerDied","Data":"ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269"}
Nov 03 22:23:39.962121 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.962029    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c56bd6bd-82w6k" event={"ID":"789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3","Type":"ContainerDied","Data":"3b00ce6e5e5dc82bd9429956df4ea8a97fadea907ef3efce57e2974c925cd8f1"}
Nov 03 22:23:39.962121 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.962048    5647 scope.go:122] "RemoveContainer" containerID="ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269"
Nov 03 22:23:39.970522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.970504    5647 scope.go:122] "RemoveContainer" containerID="ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269"
Nov 03 22:23:39.970820 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:23:39.970800    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269\": container with ID starting with ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269 not found: ID does not exist" containerID="ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269"
Nov 03 22:23:39.970891 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:39.970829    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269"} err="failed to get container status \"ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269\": rpc error: code = NotFound desc = could not find container \"ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269\": container with ID starting with ef103ce800cdd34a84ee802a42887832db02279f9b80ea16ac0ebec409677269 not found: ID does not exist"
Nov 03 22:23:40.123957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:40.123918    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c56bd6bd-82w6k"]
Nov 03 22:23:40.173502 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:40.173464    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59c56bd6bd-82w6k"]
Nov 03 22:23:40.693486 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:40.693449    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" path="/var/lib/kubelet/pods/789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3/volumes"
Nov 03 22:23:40.842076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:40.842027    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:40.842269 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:40.842104    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:40.847953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:40.847927    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:40.971236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:40.971208    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:23:41.119854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:23:41.119827    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f8cf5585d-7jxcg"]
Nov 03 22:23:46.041256 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:23:46.041210    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:24:06.140800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.140762    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-f8cf5585d-7jxcg" podUID="0929767e-35b2-4f25-86b1-0d5b197be321" containerName="console" containerID="cri-o://4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0" gracePeriod=15
Nov 03 22:24:06.390631 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.390606    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:24:06.441808 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.441736    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-oauth-serving-cert\") pod \"0929767e-35b2-4f25-86b1-0d5b197be321\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") "
Nov 03 22:24:06.441808 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.441771    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0929767e-35b2-4f25-86b1-0d5b197be321-kube-api-access-xk4k5\" (UniqueName: \"kubernetes.io/projected/0929767e-35b2-4f25-86b1-0d5b197be321-kube-api-access-xk4k5\") pod \"0929767e-35b2-4f25-86b1-0d5b197be321\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") "
Nov 03 22:24:06.441808 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.441797    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-serving-cert\") pod \"0929767e-35b2-4f25-86b1-0d5b197be321\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") "
Nov 03 22:24:06.441808 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.441814    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-service-ca\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-service-ca\") pod \"0929767e-35b2-4f25-86b1-0d5b197be321\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") "
Nov 03 22:24:06.442108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.441904    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-trusted-ca-bundle\") pod \"0929767e-35b2-4f25-86b1-0d5b197be321\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") "
Nov 03 22:24:06.442108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.441955    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-oauth-config\") pod \"0929767e-35b2-4f25-86b1-0d5b197be321\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") "
Nov 03 22:24:06.442108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.441984    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0929767e-35b2-4f25-86b1-0d5b197be321-tmp\" (UniqueName: \"kubernetes.io/empty-dir/0929767e-35b2-4f25-86b1-0d5b197be321-tmp\") pod \"0929767e-35b2-4f25-86b1-0d5b197be321\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") "
Nov 03 22:24:06.442108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442050    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-console-config\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-console-config\") pod \"0929767e-35b2-4f25-86b1-0d5b197be321\" (UID: \"0929767e-35b2-4f25-86b1-0d5b197be321\") "
Nov 03 22:24:06.442317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442191    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-oauth-serving-cert" pod "0929767e-35b2-4f25-86b1-0d5b197be321" (UID: "0929767e-35b2-4f25-86b1-0d5b197be321"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:06.442317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442248    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0929767e-35b2-4f25-86b1-0d5b197be321-tmp" pod "0929767e-35b2-4f25-86b1-0d5b197be321" (UID: "0929767e-35b2-4f25-86b1-0d5b197be321"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:24:06.442317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442255    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-service-ca" pod "0929767e-35b2-4f25-86b1-0d5b197be321" (UID: "0929767e-35b2-4f25-86b1-0d5b197be321"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:06.442455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442392    5647 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-oauth-serving-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:24:06.442455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442410    5647 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:24:06.442455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442421    5647 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0929767e-35b2-4f25-86b1-0d5b197be321-tmp\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:24:06.443028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442495    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-console-config" pod "0929767e-35b2-4f25-86b1-0d5b197be321" (UID: "0929767e-35b2-4f25-86b1-0d5b197be321"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:06.443028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.442521    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-trusted-ca-bundle" pod "0929767e-35b2-4f25-86b1-0d5b197be321" (UID: "0929767e-35b2-4f25-86b1-0d5b197be321"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:24:06.444252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.444223    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-serving-cert" pod "0929767e-35b2-4f25-86b1-0d5b197be321" (UID: "0929767e-35b2-4f25-86b1-0d5b197be321"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:06.444350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.444277    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0929767e-35b2-4f25-86b1-0d5b197be321-kube-api-access-xk4k5" pod "0929767e-35b2-4f25-86b1-0d5b197be321" (UID: "0929767e-35b2-4f25-86b1-0d5b197be321"). InnerVolumeSpecName "kube-api-access-xk4k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:24:06.444350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.444281    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-oauth-config" pod "0929767e-35b2-4f25-86b1-0d5b197be321" (UID: "0929767e-35b2-4f25-86b1-0d5b197be321"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:24:06.542987 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.542950    5647 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-console-config\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:24:06.542987 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.542980    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk4k5\" (UniqueName: \"kubernetes.io/projected/0929767e-35b2-4f25-86b1-0d5b197be321-kube-api-access-xk4k5\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:24:06.542987 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.542994    5647 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-serving-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:24:06.543206 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.543006    5647 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0929767e-35b2-4f25-86b1-0d5b197be321-trusted-ca-bundle\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:24:06.543206 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:06.543018    5647 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0929767e-35b2-4f25-86b1-0d5b197be321-console-oauth-config\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:24:07.062030 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.061990    5647 generic.go:356] "Generic (PLEG): container finished" podID="0929767e-35b2-4f25-86b1-0d5b197be321" containerID="4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0" exitCode=0
Nov 03 22:24:07.062231 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.062055    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f8cf5585d-7jxcg"
Nov 03 22:24:07.062231 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.062085    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8cf5585d-7jxcg" event={"ID":"0929767e-35b2-4f25-86b1-0d5b197be321","Type":"ContainerDied","Data":"4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0"}
Nov 03 22:24:07.062231 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.062171    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f8cf5585d-7jxcg" event={"ID":"0929767e-35b2-4f25-86b1-0d5b197be321","Type":"ContainerDied","Data":"8bc518cb0100d6ab75154be638daa2c1f87d2d253661f979e6c730466f83edec"}
Nov 03 22:24:07.062231 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.062197    5647 scope.go:122] "RemoveContainer" containerID="4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0"
Nov 03 22:24:07.070054 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.070028    5647 scope.go:122] "RemoveContainer" containerID="4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0"
Nov 03 22:24:07.070339 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:24:07.070319    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0\": container with ID starting with 4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0 not found: ID does not exist" containerID="4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0"
Nov 03 22:24:07.070385 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.070348    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0"} err="failed to get container status \"4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0\": rpc error: code = NotFound desc = could not find container \"4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0\": container with ID starting with 4839d21f9801cebf37579e3b5a24d37fbb7285c7dc18bd140be3a2f4631285b0 not found: ID does not exist"
Nov 03 22:24:07.223711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.223662    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f8cf5585d-7jxcg"]
Nov 03 22:24:07.272925 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:07.272896    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f8cf5585d-7jxcg"]
Nov 03 22:24:08.693183 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:08.693153    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0929767e-35b2-4f25-86b1-0d5b197be321" path="/var/lib/kubelet/pods/0929767e-35b2-4f25-86b1-0d5b197be321/volumes"
Nov 03 22:24:16.041134 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:24:16.041096    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:24:42.246872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.246836    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c79bb57c-2txcn"]
Nov 03 22:24:42.247300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.247153    5647 cpu_manager.go:414] "containerMap: removing container" podUID="0929767e-35b2-4f25-86b1-0d5b197be321" containerName="console"
Nov 03 22:24:42.247300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.247167    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0929767e-35b2-4f25-86b1-0d5b197be321" containerName="console"
Nov 03 22:24:42.247300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.247191    5647 cpu_manager.go:414] "containerMap: removing container" podUID="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" containerName="console"
Nov 03 22:24:42.247300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.247196    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" containerName="console"
Nov 03 22:24:42.247300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.247254    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="0929767e-35b2-4f25-86b1-0d5b197be321" containerName="console"
Nov 03 22:24:42.247300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.247267    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="789cc7c0-4a41-4117-a79f-a2c9ec5b7ec3" containerName="console"
Nov 03 22:24:42.250039 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.250019    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.334401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.334373    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-trusted-ca-bundle\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.334573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.334413    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd8vt\" (UniqueName: \"kubernetes.io/projected/537edf29-75ad-46a9-af08-e0457769a4bd-kube-api-access-cd8vt\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.334573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.334442    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-oauth-serving-cert\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.334573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.334555    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-console-config\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.334744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.334606    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/537edf29-75ad-46a9-af08-e0457769a4bd-tmp\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.334744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.334641    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-service-ca\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.334744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.334687    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/537edf29-75ad-46a9-af08-e0457769a4bd-console-serving-cert\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.334876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.334762    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/537edf29-75ad-46a9-af08-e0457769a4bd-console-oauth-config\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.352162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.352136    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c79bb57c-2txcn"]
Nov 03 22:24:42.435869 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.435824    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/537edf29-75ad-46a9-af08-e0457769a4bd-console-oauth-config\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436079 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.435928    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-trusted-ca-bundle\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436079 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.435957    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd8vt\" (UniqueName: \"kubernetes.io/projected/537edf29-75ad-46a9-af08-e0457769a4bd-kube-api-access-cd8vt\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436079 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.435989    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-oauth-serving-cert\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.436092    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-console-config\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.436146    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/537edf29-75ad-46a9-af08-e0457769a4bd-tmp\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.436168    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-service-ca\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.436220    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/537edf29-75ad-46a9-af08-e0457769a4bd-console-serving-cert\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.436568    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/537edf29-75ad-46a9-af08-e0457769a4bd-tmp\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.436840    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-oauth-serving-cert\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.436979 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.436927    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-trusted-ca-bundle\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.437041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.436974    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-service-ca\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.437222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.437197    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/537edf29-75ad-46a9-af08-e0457769a4bd-console-config\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.438572 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.438549    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/537edf29-75ad-46a9-af08-e0457769a4bd-console-oauth-config\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.438979 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.438963    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/537edf29-75ad-46a9-af08-e0457769a4bd-console-serving-cert\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.496820 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.496788    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd8vt\" (UniqueName: \"kubernetes.io/projected/537edf29-75ad-46a9-af08-e0457769a4bd-kube-api-access-cd8vt\") pod \"console-6c79bb57c-2txcn\" (UID: \"537edf29-75ad-46a9-af08-e0457769a4bd\") " pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.560961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.560870    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:42.982163 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:42.982122    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c79bb57c-2txcn"]
Nov 03 22:24:42.985909 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:24:42.985881    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537edf29_75ad_46a9_af08_e0457769a4bd.slice/crio-f3911fdbbbd0f12e818d4bf9746bc63847ad5169cdd1bef2f7e8a31f19bfe86c WatchSource:0}: Error finding container f3911fdbbbd0f12e818d4bf9746bc63847ad5169cdd1bef2f7e8a31f19bfe86c: Status 404 returned error can't find the container with id f3911fdbbbd0f12e818d4bf9746bc63847ad5169cdd1bef2f7e8a31f19bfe86c
Nov 03 22:24:43.191453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:43.191419    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c79bb57c-2txcn" event={"ID":"537edf29-75ad-46a9-af08-e0457769a4bd","Type":"ContainerStarted","Data":"a73b7c0c8e7fe6e905c76c95ed616afc28c990bfa3d1c525c7afe9d03409b8a9"}
Nov 03 22:24:43.191633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:43.191465    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c79bb57c-2txcn" event={"ID":"537edf29-75ad-46a9-af08-e0457769a4bd","Type":"ContainerStarted","Data":"f3911fdbbbd0f12e818d4bf9746bc63847ad5169cdd1bef2f7e8a31f19bfe86c"}
Nov 03 22:24:43.309094 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:43.308993    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6c79bb57c-2txcn" podStartSLOduration=1.308976282 podStartE2EDuration="1.308976282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:24:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:24:43.308213869 +0000 UTC m=+106.748751142" watchObservedRunningTime="2026-11-03 22:24:43.308976282 +0000 UTC m=+106.749513542"
Nov 03 22:24:46.041334 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:24:46.041297    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:24:52.561435 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:52.561400    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:52.561878 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:52.561485    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:52.566216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:52.566194    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:53.229382 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:53.229353    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c79bb57c-2txcn"
Nov 03 22:24:53.397583 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:24:53.397547    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d65f9c6bb-2kr92"]
Nov 03 22:25:16.040847 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:25:16.040810    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:25:18.417639 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.417570    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7d65f9c6bb-2kr92" podUID="f2bbf368-f12f-4683-bfba-a0ccae8b4a80" containerName="console" containerID="cri-o://e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e" gracePeriod=15
Nov 03 22:25:18.669393 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.669327    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:25:18.843348 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843317    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-config\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-config\") pod \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") "
Nov 03 22:25:18.843534 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843360    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-trusted-ca-bundle\") pod \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") "
Nov 03 22:25:18.843534 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843402    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-tmp\") pod \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") "
Nov 03 22:25:18.843534 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843444    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-service-ca\") pod \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") "
Nov 03 22:25:18.843731 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843606    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-oauth-config\") pod \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") "
Nov 03 22:25:18.843731 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843696    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-serving-cert\") pod \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") "
Nov 03 22:25:18.843840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843763    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-oauth-serving-cert\") pod \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") "
Nov 03 22:25:18.843840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843772    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-tmp" pod "f2bbf368-f12f-4683-bfba-a0ccae8b4a80" (UID: "f2bbf368-f12f-4683-bfba-a0ccae8b4a80"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:25:18.843840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843788    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-config" pod "f2bbf368-f12f-4683-bfba-a0ccae8b4a80" (UID: "f2bbf368-f12f-4683-bfba-a0ccae8b4a80"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:25:18.843840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843803    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-kube-api-access-dc7wx\" (UniqueName: \"kubernetes.io/projected/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-kube-api-access-dc7wx\") pod \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\" (UID: \"f2bbf368-f12f-4683-bfba-a0ccae8b4a80\") "
Nov 03 22:25:18.844025 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843915    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-service-ca" pod "f2bbf368-f12f-4683-bfba-a0ccae8b4a80" (UID: "f2bbf368-f12f-4683-bfba-a0ccae8b4a80"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:25:18.844081 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.843927    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-trusted-ca-bundle" pod "f2bbf368-f12f-4683-bfba-a0ccae8b4a80" (UID: "f2bbf368-f12f-4683-bfba-a0ccae8b4a80"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:25:18.844173 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.844153    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-oauth-serving-cert" pod "f2bbf368-f12f-4683-bfba-a0ccae8b4a80" (UID: "f2bbf368-f12f-4683-bfba-a0ccae8b4a80"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:25:18.844233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.844174    5647 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-tmp\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:25:18.844233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.844196    5647 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:25:18.844233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.844211    5647 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-config\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:25:18.844233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.844225    5647 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-trusted-ca-bundle\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:25:18.845921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.845890    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-oauth-config" pod "f2bbf368-f12f-4683-bfba-a0ccae8b4a80" (UID: "f2bbf368-f12f-4683-bfba-a0ccae8b4a80"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:25:18.846010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.845985    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-serving-cert" pod "f2bbf368-f12f-4683-bfba-a0ccae8b4a80" (UID: "f2bbf368-f12f-4683-bfba-a0ccae8b4a80"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:25:18.846094 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.846075    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-kube-api-access-dc7wx" pod "f2bbf368-f12f-4683-bfba-a0ccae8b4a80" (UID: "f2bbf368-f12f-4683-bfba-a0ccae8b4a80"). InnerVolumeSpecName "kube-api-access-dc7wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:25:18.945397 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.945300    5647 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-oauth-config\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:25:18.945397 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.945345    5647 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-console-serving-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:25:18.945397 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.945356    5647 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-oauth-serving-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:25:18.945397 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:18.945364    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc7wx\" (UniqueName: \"kubernetes.io/projected/f2bbf368-f12f-4683-bfba-a0ccae8b4a80-kube-api-access-dc7wx\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:25:19.323533 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.323494    5647 generic.go:356] "Generic (PLEG): container finished" podID="f2bbf368-f12f-4683-bfba-a0ccae8b4a80" containerID="e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e" exitCode=0
Nov 03 22:25:19.323750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.323551    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d65f9c6bb-2kr92"
Nov 03 22:25:19.323750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.323562    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d65f9c6bb-2kr92" event={"ID":"f2bbf368-f12f-4683-bfba-a0ccae8b4a80","Type":"ContainerDied","Data":"e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e"}
Nov 03 22:25:19.323750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.323632    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d65f9c6bb-2kr92" event={"ID":"f2bbf368-f12f-4683-bfba-a0ccae8b4a80","Type":"ContainerDied","Data":"59335967588e7c9679a0b98664178f47a19229a71d74b3448686142dea3d356b"}
Nov 03 22:25:19.323750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.323653    5647 scope.go:122] "RemoveContainer" containerID="e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e"
Nov 03 22:25:19.332207 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.332191    5647 scope.go:122] "RemoveContainer" containerID="e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e"
Nov 03 22:25:19.332473 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:25:19.332455    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e\": container with ID starting with e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e not found: ID does not exist" containerID="e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e"
Nov 03 22:25:19.332514 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.332484    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e"} err="failed to get container status \"e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e\": rpc error: code = NotFound desc = could not find container \"e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e\": container with ID starting with e86a650ac49a839d5d9d16d45238b216518d2f9cd3f541e55b8b895bf4de7e0e not found: ID does not exist"
Nov 03 22:25:19.488885 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.488851    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d65f9c6bb-2kr92"]
Nov 03 22:25:19.538299 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:19.538269    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d65f9c6bb-2kr92"]
Nov 03 22:25:20.693576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:25:20.693541    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2bbf368-f12f-4683-bfba-a0ccae8b4a80" path="/var/lib/kubelet/pods/f2bbf368-f12f-4683-bfba-a0ccae8b4a80/volumes"
Nov 03 22:25:46.041102 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:25:46.041061    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:26:16.041044 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:26:16.040961    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:26:46.041496 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:26:46.041455    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:26:53.857714 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.857680    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"]
Nov 03 22:26:53.858141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.857990    5647 cpu_manager.go:414] "containerMap: removing container" podUID="f2bbf368-f12f-4683-bfba-a0ccae8b4a80" containerName="console"
Nov 03 22:26:53.858141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.858002    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2bbf368-f12f-4683-bfba-a0ccae8b4a80" containerName="console"
Nov 03 22:26:53.858141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.858070    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2bbf368-f12f-4683-bfba-a0ccae8b4a80" containerName="console"
Nov 03 22:26:53.860777 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.860761    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:53.962357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.962313    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 03 22:26:53.962357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.962323    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-rsbsr\""
Nov 03 22:26:53.962357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.962352    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 03 22:26:53.963242 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.963223    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"]
Nov 03 22:26:53.988834 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.988798    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lnt5\" (UniqueName: \"kubernetes.io/projected/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-kube-api-access-6lnt5\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:53.988991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.988861    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:53.988991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:53.988946    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:54.090232 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.090201    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:54.090431 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.090261    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lnt5\" (UniqueName: \"kubernetes.io/projected/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-kube-api-access-6lnt5\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:54.090431 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.090300    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:54.090611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.090592    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:54.090687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.090619    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:54.136121 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.136047    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"]
Nov 03 22:26:54.139109 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.139091    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.150829 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.150801    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lnt5\" (UniqueName: \"kubernetes.io/projected/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-kube-api-access-6lnt5\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:54.175599 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.175577    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:26:54.191698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.191648    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkm7w\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-kube-api-access-kkm7w\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.191830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.191809    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.191897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.191857    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.191964 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.191943    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.192016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.191984    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.192066 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.192044    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e810ddd3-8e80-499d-a409-12a8d143dc79-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.192107 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.192076    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.192631 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.192613    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 22:26:54.192853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.192837    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 03 22:26:54.193644 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.193609    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 03 22:26:54.193644 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.193636    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 03 22:26:54.193883 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.193658    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-69szt\""
Nov 03 22:26:54.196459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.196441    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 22:26:54.201962 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.201939    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"]
Nov 03 22:26:54.293510 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.293471    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.293701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.293517    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.293701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.293562    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.293701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.293587    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e810ddd3-8e80-499d-a409-12a8d143dc79-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.293701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.293634    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.293902 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.293716    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkm7w\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-kube-api-access-kkm7w\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.293902 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.293782    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.294297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.294269    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.296309 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.296288    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.296599 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.296582    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e810ddd3-8e80-499d-a409-12a8d143dc79-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.296680 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.296617    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.296818 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.296781    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 03 22:26:54.306926 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.306901    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.354769 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.354738    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.354769 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.354765    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkm7w\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-kube-api-access-kkm7w\") pod \"istiod-openshift-gateway-7876bfcb74-gwwlg\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.449964 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.449879    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:54.546187 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.546148    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"]
Nov 03 22:26:54.550168 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:26:54.550141    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6c19a5_097e_4e47_bd2f_2a82bcb43827.slice/crio-cade969f0e8ade5ed89ea0adbf136968081f96e3a1734d622effbf613f3cfff5 WatchSource:0}: Error finding container cade969f0e8ade5ed89ea0adbf136968081f96e3a1734d622effbf613f3cfff5: Status 404 returned error can't find the container with id cade969f0e8ade5ed89ea0adbf136968081f96e3a1734d622effbf613f3cfff5
Nov 03 22:26:54.654151 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.654121    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48" event={"ID":"8b6c19a5-097e-4e47-bd2f-2a82bcb43827","Type":"ContainerStarted","Data":"cade969f0e8ade5ed89ea0adbf136968081f96e3a1734d622effbf613f3cfff5"}
Nov 03 22:26:54.827267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:54.827186    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"]
Nov 03 22:26:54.831551 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:26:54.831528    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode810ddd3_8e80_499d_a409_12a8d143dc79.slice/crio-d92480da5129144f0f9e3937365667a9d700a0f92999dfa0b8440563fe9df8dd WatchSource:0}: Error finding container d92480da5129144f0f9e3937365667a9d700a0f92999dfa0b8440563fe9df8dd: Status 404 returned error can't find the container with id d92480da5129144f0f9e3937365667a9d700a0f92999dfa0b8440563fe9df8dd
Nov 03 22:26:55.659266 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:55.659221    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg" event={"ID":"e810ddd3-8e80-499d-a409-12a8d143dc79","Type":"ContainerStarted","Data":"d92480da5129144f0f9e3937365667a9d700a0f92999dfa0b8440563fe9df8dd"}
Nov 03 22:26:58.611905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:58.611873    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:26:58.612280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:58.611934    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:26:59.681710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:59.681648    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg" event={"ID":"e810ddd3-8e80-499d-a409-12a8d143dc79","Type":"ContainerStarted","Data":"fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b"}
Nov 03 22:26:59.682155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:59.681845    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:26:59.683790 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:59.683765    5647 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-gwwlg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:26:59.683938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:26:59.683810    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg" podUID="e810ddd3-8e80-499d-a409-12a8d143dc79" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:27:00.686809 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:00.686775    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:27:00.860292 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:00.860242    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg" podStartSLOduration=3.082690946 podStartE2EDuration="6.860225396s" totalImagesPullingTime="3.77753445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:26:59.834247779 +0000 UTC m=+243.274785039" watchObservedRunningTime="2026-11-03 22:27:00.860225396 +0000 UTC m=+244.300762646"
Nov 03 22:27:01.690888 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:01.690858    5647 generic.go:356] "Generic (PLEG): container finished" podID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerID="dea631567466eff010e76ac2b39ef6c9f8ec886abae2df734c050c77febf1abf" exitCode=0
Nov 03 22:27:01.691295 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:01.690951    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48" event={"ID":"8b6c19a5-097e-4e47-bd2f-2a82bcb43827","Type":"ContainerDied","Data":"dea631567466eff010e76ac2b39ef6c9f8ec886abae2df734c050c77febf1abf"}
Nov 03 22:27:02.696781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:02.696748    5647 generic.go:356] "Generic (PLEG): container finished" podID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerID="0da6c497b4331dddd9859b705ee2711f0d97fd372c46db15647896d95b5eca06" exitCode=0
Nov 03 22:27:02.697184 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:02.696834    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48" event={"ID":"8b6c19a5-097e-4e47-bd2f-2a82bcb43827","Type":"ContainerDied","Data":"0da6c497b4331dddd9859b705ee2711f0d97fd372c46db15647896d95b5eca06"}
Nov 03 22:27:09.726183 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:09.726140    5647 generic.go:356] "Generic (PLEG): container finished" podID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerID="d140ee678e467d86c5a6208bae370a44ae45a68aae4f100d373b4fc1d6041dcf" exitCode=0
Nov 03 22:27:09.726573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:09.726190    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48" event={"ID":"8b6c19a5-097e-4e47-bd2f-2a82bcb43827","Type":"ContainerDied","Data":"d140ee678e467d86c5a6208bae370a44ae45a68aae4f100d373b4fc1d6041dcf"}
Nov 03 22:27:10.854195 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:10.854163    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:27:10.953508 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:10.953471    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-kube-api-access-6lnt5\" (UniqueName: \"kubernetes.io/projected/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-kube-api-access-6lnt5\") pod \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") "
Nov 03 22:27:10.953719 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:10.953621    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-bundle\") pod \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") "
Nov 03 22:27:10.953719 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:10.953650    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-util\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-util\") pod \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\" (UID: \"8b6c19a5-097e-4e47-bd2f-2a82bcb43827\") "
Nov 03 22:27:10.954221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:10.954191    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-bundle" pod "8b6c19a5-097e-4e47-bd2f-2a82bcb43827" (UID: "8b6c19a5-097e-4e47-bd2f-2a82bcb43827"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:27:10.955930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:10.955898    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-kube-api-access-6lnt5" pod "8b6c19a5-097e-4e47-bd2f-2a82bcb43827" (UID: "8b6c19a5-097e-4e47-bd2f-2a82bcb43827"). InnerVolumeSpecName "kube-api-access-6lnt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:27:10.957683 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:10.957641    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-util" pod "8b6c19a5-097e-4e47-bd2f-2a82bcb43827" (UID: "8b6c19a5-097e-4e47-bd2f-2a82bcb43827"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:27:11.055310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:11.055220    5647 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-bundle\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:27:11.055310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:11.055252    5647 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-util\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:27:11.055310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:11.055263    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lnt5\" (UniqueName: \"kubernetes.io/projected/8b6c19a5-097e-4e47-bd2f-2a82bcb43827-kube-api-access-6lnt5\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:27:11.734850 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:11.734814    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48" event={"ID":"8b6c19a5-097e-4e47-bd2f-2a82bcb43827","Type":"ContainerDied","Data":"cade969f0e8ade5ed89ea0adbf136968081f96e3a1734d622effbf613f3cfff5"}
Nov 03 22:27:11.734850 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:11.734833    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71v4r48"
Nov 03 22:27:11.735062 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:11.734863    5647 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cade969f0e8ade5ed89ea0adbf136968081f96e3a1734d622effbf613f3cfff5"
Nov 03 22:27:16.041626 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:27:16.041590    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:27:38.365648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.365564    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"]
Nov 03 22:27:38.366044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.365898    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerName="extract"
Nov 03 22:27:38.366044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.365911    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerName="extract"
Nov 03 22:27:38.366044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.365924    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerName="util"
Nov 03 22:27:38.366044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.365929    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerName="util"
Nov 03 22:27:38.366044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.365942    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerName="pull"
Nov 03 22:27:38.366044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.365947    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerName="pull"
Nov 03 22:27:38.366044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.366015    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b6c19a5-097e-4e47-bd2f-2a82bcb43827" containerName="extract"
Nov 03 22:27:38.367920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.367904    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.447407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.447378    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 03 22:27:38.466923 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.466889    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b51abea1-c4c2-40d1-adab-a4516346aad4-manifests\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.467063 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.466930    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51abea1-c4c2-40d1-adab-a4516346aad4-tmp\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.467063 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.466948    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.467063 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.466966    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nfdx\" (UniqueName: \"kubernetes.io/projected/b51abea1-c4c2-40d1-adab-a4516346aad4-kube-api-access-9nfdx\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.467167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.467067    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.467167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.467117    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.473898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.473878    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 03 22:27:38.474034 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.473878    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-6hr8v\""
Nov 03 22:27:38.474034 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.473915    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 03 22:27:38.474034 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.473929    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 03 22:27:38.474034 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.473993    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 03 22:27:38.477882 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.477858    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"]
Nov 03 22:27:38.568544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.568516    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.568726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.568561    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.568726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.568600    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b51abea1-c4c2-40d1-adab-a4516346aad4-manifests\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.568726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.568619    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51abea1-c4c2-40d1-adab-a4516346aad4-tmp\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.568726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.568637    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.568726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.568652    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nfdx\" (UniqueName: \"kubernetes.io/projected/b51abea1-c4c2-40d1-adab-a4516346aad4-kube-api-access-9nfdx\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.569120 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.569088    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b51abea1-c4c2-40d1-adab-a4516346aad4-manifests\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.571388 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.571358    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-metrics-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.571476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.571391    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.571476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.571441    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51abea1-c4c2-40d1-adab-a4516346aad4-tmp\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.571786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.571766    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b51abea1-c4c2-40d1-adab-a4516346aad4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.629529 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.629460    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nfdx\" (UniqueName: \"kubernetes.io/projected/b51abea1-c4c2-40d1-adab-a4516346aad4-kube-api-access-9nfdx\") pod \"opendatahub-operator-controller-manager-6b88f79f5c-gd2c6\" (UID: \"b51abea1-c4c2-40d1-adab-a4516346aad4\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:38.680807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:38.680778    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:39.176192 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:39.176157    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"]
Nov 03 22:27:39.179111 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:27:39.179087    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb51abea1_c4c2_40d1_adab_a4516346aad4.slice/crio-fb962fea6ea28403a54d05531cb2ee556d2f2b6d3fa6e31bee3c8446adb5dd14 WatchSource:0}: Error finding container fb962fea6ea28403a54d05531cb2ee556d2f2b6d3fa6e31bee3c8446adb5dd14: Status 404 returned error can't find the container with id fb962fea6ea28403a54d05531cb2ee556d2f2b6d3fa6e31bee3c8446adb5dd14
Nov 03 22:27:39.837744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:39.837710    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6" event={"ID":"b51abea1-c4c2-40d1-adab-a4516346aad4","Type":"ContainerStarted","Data":"fb962fea6ea28403a54d05531cb2ee556d2f2b6d3fa6e31bee3c8446adb5dd14"}
Nov 03 22:27:42.851939 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:42.851854    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6" event={"ID":"b51abea1-c4c2-40d1-adab-a4516346aad4","Type":"ContainerStarted","Data":"e9b281d333336b8c2d1d0d58a221df6fa68c1fc88bf5d64afa424d62924ac424"}
Nov 03 22:27:43.856930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:43.856892    5647 generic.go:356] "Generic (PLEG): container finished" podID="b51abea1-c4c2-40d1-adab-a4516346aad4" containerID="e9b281d333336b8c2d1d0d58a221df6fa68c1fc88bf5d64afa424d62924ac424" exitCode=0
Nov 03 22:27:43.857357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:43.856984    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6" event={"ID":"b51abea1-c4c2-40d1-adab-a4516346aad4","Type":"ContainerDied","Data":"e9b281d333336b8c2d1d0d58a221df6fa68c1fc88bf5d64afa424d62924ac424"}
Nov 03 22:27:44.864167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:44.864130    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6" event={"ID":"b51abea1-c4c2-40d1-adab-a4516346aad4","Type":"ContainerStarted","Data":"7de888a386b19551ca181c7826f0fbf013263f1e3c19c5bba3d00332b6524c38"}
Nov 03 22:27:44.864658 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:44.864234    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:45.103023 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:45.102966    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6" podStartSLOduration=2.490141542 podStartE2EDuration="7.102948369s" totalImagesPullingTime="4.119879409s" totalInitContainerRuntime="492.927418ms" isStatefulPod=false podCreationTimestamp="2026-11-03 22:27:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:27:45.053531488 +0000 UTC m=+288.494068771" watchObservedRunningTime="2026-11-03 22:27:45.102948369 +0000 UTC m=+288.543485628"
Nov 03 22:27:46.052956 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:27:46.052912    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:27:55.871134 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:55.871104    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6b88f79f5c-gd2c6"
Nov 03 22:27:56.717687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:56.717649    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:27:56.717884 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:27:56.717654    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:28:02.755484 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.755453    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"]
Nov 03 22:28:02.763859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.763842    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"
Nov 03 22:28:02.821772 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.821738    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"]
Nov 03 22:28:02.870130 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.870094    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9f4pf\""
Nov 03 22:28:02.870316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.870206    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 03 22:28:02.870316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.870227    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 03 22:28:02.883597 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.883570    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dwm4\" (UniqueName: \"kubernetes.io/projected/86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba-kube-api-access-6dwm4\") pod \"cert-manager-cainjector-5c57bbd4fc-wvcds\" (UID: \"86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"
Nov 03 22:28:02.883704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.883608    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wvcds\" (UID: \"86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"
Nov 03 22:28:02.984568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.984534    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dwm4\" (UniqueName: \"kubernetes.io/projected/86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba-kube-api-access-6dwm4\") pod \"cert-manager-cainjector-5c57bbd4fc-wvcds\" (UID: \"86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"
Nov 03 22:28:02.984821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:02.984585    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wvcds\" (UID: \"86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"
Nov 03 22:28:03.050446 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:03.050371    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wvcds\" (UID: \"86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"
Nov 03 22:28:03.050572 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:03.050492    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dwm4\" (UniqueName: \"kubernetes.io/projected/86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba-kube-api-access-6dwm4\") pod \"cert-manager-cainjector-5c57bbd4fc-wvcds\" (UID: \"86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"
Nov 03 22:28:03.083280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:03.083247    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"
Nov 03 22:28:03.453946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:03.453859    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds"]
Nov 03 22:28:03.457201 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:28:03.457174    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86fb8bb5_8666_4d49_aeaa_d8c70d2c9bba.slice/crio-6e1d5ed69b86e7e09f99f442dd5e09aac4370225f2f10f3415dba564c8c17aeb WatchSource:0}: Error finding container 6e1d5ed69b86e7e09f99f442dd5e09aac4370225f2f10f3415dba564c8c17aeb: Status 404 returned error can't find the container with id 6e1d5ed69b86e7e09f99f442dd5e09aac4370225f2f10f3415dba564c8c17aeb
Nov 03 22:28:03.459339 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:03.459324    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:28:03.938835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:03.938801    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds" event={"ID":"86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba","Type":"ContainerStarted","Data":"6e1d5ed69b86e7e09f99f442dd5e09aac4370225f2f10f3415dba564c8c17aeb"}
Nov 03 22:28:07.959217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:07.959183    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds" event={"ID":"86fb8bb5-8666-4d49-aeaa-d8c70d2c9bba","Type":"ContainerStarted","Data":"10ab2a790b692787c3000a46cc99d5fa1496f46737de07154f557eb26ec29017"}
Nov 03 22:28:08.078073 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:08.078014    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wvcds" podStartSLOduration=2.158482734 podStartE2EDuration="6.077999884s" totalImagesPullingTime="3.91951715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:28:08.076883599 +0000 UTC m=+311.517420914" watchObservedRunningTime="2026-11-03 22:28:08.077999884 +0000 UTC m=+311.518537137"
Nov 03 22:28:09.530151 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.530115    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"]
Nov 03 22:28:09.533616 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.533596    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.644692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.644629    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 03 22:28:09.644898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.644739    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sqnzz\""
Nov 03 22:28:09.644898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.644821    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 03 22:28:09.645014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.644991    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82ca6b39-a72a-4632-9342-10eca3d67829-tmp\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.645061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.645031    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82ca6b39-a72a-4632-9342-10eca3d67829-tls-certs\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.645109 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.645056    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5pjl\" (UniqueName: \"kubernetes.io/projected/82ca6b39-a72a-4632-9342-10eca3d67829-kube-api-access-h5pjl\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.645932 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.645908    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"]
Nov 03 22:28:09.745993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.745956    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82ca6b39-a72a-4632-9342-10eca3d67829-tmp\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.746199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.746011    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82ca6b39-a72a-4632-9342-10eca3d67829-tls-certs\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.746199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.746035    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5pjl\" (UniqueName: \"kubernetes.io/projected/82ca6b39-a72a-4632-9342-10eca3d67829-kube-api-access-h5pjl\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.748433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.748404    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82ca6b39-a72a-4632-9342-10eca3d67829-tmp\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.748638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.748617    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82ca6b39-a72a-4632-9342-10eca3d67829-tls-certs\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.807216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.807142    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5pjl\" (UniqueName: \"kubernetes.io/projected/82ca6b39-a72a-4632-9342-10eca3d67829-kube-api-access-h5pjl\") pod \"kube-auth-proxy-586b8ddf6d-h9m6s\" (UID: \"82ca6b39-a72a-4632-9342-10eca3d67829\") " pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:09.850096 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:09.850066    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"
Nov 03 22:28:10.252748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:10.252714    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s"]
Nov 03 22:28:10.256053 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:28:10.256026    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ca6b39_a72a_4632_9342_10eca3d67829.slice/crio-b96ac8a3c637d9eff2807706dc120d3e5248e65ddceac72368b55a0a516eb79a WatchSource:0}: Error finding container b96ac8a3c637d9eff2807706dc120d3e5248e65ddceac72368b55a0a516eb79a: Status 404 returned error can't find the container with id b96ac8a3c637d9eff2807706dc120d3e5248e65ddceac72368b55a0a516eb79a
Nov 03 22:28:10.975068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:10.975033    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s" event={"ID":"82ca6b39-a72a-4632-9342-10eca3d67829","Type":"ContainerStarted","Data":"b96ac8a3c637d9eff2807706dc120d3e5248e65ddceac72368b55a0a516eb79a"}
Nov 03 22:28:11.871940 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:11.871900    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-j5csb"]
Nov 03 22:28:11.875301 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:11.875281    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-j5csb"
Nov 03 22:28:11.931839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:11.931802    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-j5csb"]
Nov 03 22:28:11.967935 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:11.967898    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqcn8\" (UniqueName: \"kubernetes.io/projected/31c9c143-5798-4346-b87e-99628ccf7c62-kube-api-access-zqcn8\") pod \"cert-manager-597b95cb84-j5csb\" (UID: \"31c9c143-5798-4346-b87e-99628ccf7c62\") " pod="cert-manager/cert-manager-597b95cb84-j5csb"
Nov 03 22:28:11.968101 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:11.967949    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31c9c143-5798-4346-b87e-99628ccf7c62-bound-sa-token\") pod \"cert-manager-597b95cb84-j5csb\" (UID: \"31c9c143-5798-4346-b87e-99628ccf7c62\") " pod="cert-manager/cert-manager-597b95cb84-j5csb"
Nov 03 22:28:11.999432 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:11.999409    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lkmvb\""
Nov 03 22:28:12.068998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:12.068956    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqcn8\" (UniqueName: \"kubernetes.io/projected/31c9c143-5798-4346-b87e-99628ccf7c62-kube-api-access-zqcn8\") pod \"cert-manager-597b95cb84-j5csb\" (UID: \"31c9c143-5798-4346-b87e-99628ccf7c62\") " pod="cert-manager/cert-manager-597b95cb84-j5csb"
Nov 03 22:28:12.069165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:12.069006    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31c9c143-5798-4346-b87e-99628ccf7c62-bound-sa-token\") pod \"cert-manager-597b95cb84-j5csb\" (UID: \"31c9c143-5798-4346-b87e-99628ccf7c62\") " pod="cert-manager/cert-manager-597b95cb84-j5csb"
Nov 03 22:28:12.130219 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:12.130153    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31c9c143-5798-4346-b87e-99628ccf7c62-bound-sa-token\") pod \"cert-manager-597b95cb84-j5csb\" (UID: \"31c9c143-5798-4346-b87e-99628ccf7c62\") " pod="cert-manager/cert-manager-597b95cb84-j5csb"
Nov 03 22:28:12.130371 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:12.130292    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqcn8\" (UniqueName: \"kubernetes.io/projected/31c9c143-5798-4346-b87e-99628ccf7c62-kube-api-access-zqcn8\") pod \"cert-manager-597b95cb84-j5csb\" (UID: \"31c9c143-5798-4346-b87e-99628ccf7c62\") " pod="cert-manager/cert-manager-597b95cb84-j5csb"
Nov 03 22:28:12.189410 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:12.189372    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-j5csb"
Nov 03 22:28:12.573042 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:12.573010    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-j5csb"]
Nov 03 22:28:12.588823 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:28:12.588795    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31c9c143_5798_4346_b87e_99628ccf7c62.slice/crio-717197cda5017216e980c0ea0cde29379547f465608dbb3f5127df209ea2bffc WatchSource:0}: Error finding container 717197cda5017216e980c0ea0cde29379547f465608dbb3f5127df209ea2bffc: Status 404 returned error can't find the container with id 717197cda5017216e980c0ea0cde29379547f465608dbb3f5127df209ea2bffc
Nov 03 22:28:12.989788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:12.989694    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-j5csb" event={"ID":"31c9c143-5798-4346-b87e-99628ccf7c62","Type":"ContainerStarted","Data":"784975ae49902f75fe910e10e905bd7bfd2f3fab6f217889dc86ed6d6fc6755c"}
Nov 03 22:28:12.989788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:12.989753    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-j5csb" event={"ID":"31c9c143-5798-4346-b87e-99628ccf7c62","Type":"ContainerStarted","Data":"717197cda5017216e980c0ea0cde29379547f465608dbb3f5127df209ea2bffc"}
Nov 03 22:28:13.108499 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.108431    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-j5csb" podStartSLOduration=2.108412043 podStartE2EDuration="2.108412043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:28:13.105965502 +0000 UTC m=+316.546502763" watchObservedRunningTime="2026-11-03 22:28:13.108412043 +0000 UTC m=+316.548949345"
Nov 03 22:28:13.686896 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.686862    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:28:13.690289 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.690273    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:13.746600 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.746571    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 03 22:28:13.746600 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.746580    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 03 22:28:13.746943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.746603    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 03 22:28:13.747202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.747177    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-plwfc\""
Nov 03 22:28:13.747957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.747934    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:28:13.784847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.784819    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/891f78e8-3535-4eea-9912-b93f9fc571c3-config-file\") pod \"limitador-limitador-7c9b87d574-4lqhk\" (UID: \"891f78e8-3535-4eea-9912-b93f9fc571c3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:13.784990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.784878    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r75tv\" (UniqueName: \"kubernetes.io/projected/891f78e8-3535-4eea-9912-b93f9fc571c3-kube-api-access-r75tv\") pod \"limitador-limitador-7c9b87d574-4lqhk\" (UID: \"891f78e8-3535-4eea-9912-b93f9fc571c3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:13.886156 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.886120    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/891f78e8-3535-4eea-9912-b93f9fc571c3-config-file\") pod \"limitador-limitador-7c9b87d574-4lqhk\" (UID: \"891f78e8-3535-4eea-9912-b93f9fc571c3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:13.886321 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.886174    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r75tv\" (UniqueName: \"kubernetes.io/projected/891f78e8-3535-4eea-9912-b93f9fc571c3-kube-api-access-r75tv\") pod \"limitador-limitador-7c9b87d574-4lqhk\" (UID: \"891f78e8-3535-4eea-9912-b93f9fc571c3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:13.886756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.886737    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/891f78e8-3535-4eea-9912-b93f9fc571c3-config-file\") pod \"limitador-limitador-7c9b87d574-4lqhk\" (UID: \"891f78e8-3535-4eea-9912-b93f9fc571c3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:13.949517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.949443    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r75tv\" (UniqueName: \"kubernetes.io/projected/891f78e8-3535-4eea-9912-b93f9fc571c3-kube-api-access-r75tv\") pod \"limitador-limitador-7c9b87d574-4lqhk\" (UID: \"891f78e8-3535-4eea-9912-b93f9fc571c3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:13.996069 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:13.996025    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s" event={"ID":"82ca6b39-a72a-4632-9342-10eca3d67829","Type":"ContainerStarted","Data":"c54a4bfb51e9c64599e16c3fa43598087a823b6bad297f10743241c895a50d7b"}
Nov 03 22:28:14.006916 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:14.006886    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:28:14.010090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:14.010066    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:14.111926 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:14.111860    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s" podStartSLOduration=2.200578842 podStartE2EDuration="5.111841564s" totalImagesPullingTime="2.911262722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:28:14.110374199 +0000 UTC m=+317.550911459" watchObservedRunningTime="2026-11-03 22:28:14.111841564 +0000 UTC m=+317.552378826"
Nov 03 22:28:14.377178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:14.377147    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:28:14.380244 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:28:14.380216    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod891f78e8_3535_4eea_9912_b93f9fc571c3.slice/crio-4e58493ecb87da451f42ae51e88f2abdc796c4cb2cf6eef1fc9d7120c734427e WatchSource:0}: Error finding container 4e58493ecb87da451f42ae51e88f2abdc796c4cb2cf6eef1fc9d7120c734427e: Status 404 returned error can't find the container with id 4e58493ecb87da451f42ae51e88f2abdc796c4cb2cf6eef1fc9d7120c734427e
Nov 03 22:28:15.001940 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:15.001900    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk" event={"ID":"891f78e8-3535-4eea-9912-b93f9fc571c3","Type":"ContainerStarted","Data":"4e58493ecb87da451f42ae51e88f2abdc796c4cb2cf6eef1fc9d7120c734427e"}
Nov 03 22:28:16.054569 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:28:16.054466    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:28:18.020263 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:18.020228    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk" event={"ID":"891f78e8-3535-4eea-9912-b93f9fc571c3","Type":"ContainerStarted","Data":"c5dd1fb663f80da8cabecfeac2b090e2ab2e90bd5d519b875603d8df9481322d"}
Nov 03 22:28:18.020724 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:18.020350    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:18.138320 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:18.138264    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk" podStartSLOduration=1.94877056 podStartE2EDuration="5.138249118s" totalImagesPullingTime="3.189478558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:28:18.137090411 +0000 UTC m=+321.577627674" watchObservedRunningTime="2026-11-03 22:28:18.138249118 +0000 UTC m=+321.578786378"
Nov 03 22:28:29.025730 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:28:29.025700    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"
Nov 03 22:28:46.041711 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:28:46.041650    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:29:16.043370 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:29:16.043293    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:29:23.093059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.093027    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"]
Nov 03 22:29:23.096451 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.096432    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.185236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.185189    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.185236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.185228    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.185452 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.185314    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccmcm\" (UniqueName: \"kubernetes.io/projected/2ff0e28c-db6e-4176-b124-a79a6f78ba51-kube-api-access-ccmcm\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.185452 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.185384    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff0e28c-db6e-4176-b124-a79a6f78ba51-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.185533 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.185467    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.185533 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.185487    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.185533 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.185509    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.198986 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.198956    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"]
Nov 03 22:29:23.285970 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.285932    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.286169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.285983    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccmcm\" (UniqueName: \"kubernetes.io/projected/2ff0e28c-db6e-4176-b124-a79a6f78ba51-kube-api-access-ccmcm\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.286169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.286022    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff0e28c-db6e-4176-b124-a79a6f78ba51-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.286169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.286047    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.286169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.286072    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.286169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.286099    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.286169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.286133    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.286877 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.286844    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.288848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.288815    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff0e28c-db6e-4176-b124-a79a6f78ba51-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.289242 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.289216    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.289355 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.289298    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.289355 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.289326    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.347067 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.346996    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccmcm\" (UniqueName: \"kubernetes.io/projected/2ff0e28c-db6e-4176-b124-a79a6f78ba51-kube-api-access-ccmcm\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.347067 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.347048    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff0e28c-db6e-4176-b124-a79a6f78ba51-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-cwdgf\" (UID: \"2ff0e28c-db6e-4176-b124-a79a6f78ba51\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.407373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.407341    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:23.827744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.827716    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"]
Nov 03 22:29:23.830131 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:29:23.830105    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff0e28c_db6e_4176_b124_a79a6f78ba51.slice/crio-c81f8b69871748c85abdce925cdb2856a73b9365b7664469fd8b0efdb887ea89 WatchSource:0}: Error finding container c81f8b69871748c85abdce925cdb2856a73b9365b7664469fd8b0efdb887ea89: Status 404 returned error can't find the container with id c81f8b69871748c85abdce925cdb2856a73b9365b7664469fd8b0efdb887ea89
Nov 03 22:29:23.832065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.832037    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:29:23.832125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:23.832094    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:29:24.296505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:24.296414    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf" event={"ID":"2ff0e28c-db6e-4176-b124-a79a6f78ba51","Type":"ContainerStarted","Data":"8d1cd8bc857ff6719146438e982f8fc4385e7392492dc42284ffe53e982713bb"}
Nov 03 22:29:24.296505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:24.296476    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf" event={"ID":"2ff0e28c-db6e-4176-b124-a79a6f78ba51","Type":"ContainerStarted","Data":"c81f8b69871748c85abdce925cdb2856a73b9365b7664469fd8b0efdb887ea89"}
Nov 03 22:29:24.297060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:24.296800    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:24.298173 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:24.298146    5647 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:29:24.298275 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:24.298196    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf" podUID="2ff0e28c-db6e-4176-b124-a79a6f78ba51" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:29:24.417012 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:24.416958    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf" podStartSLOduration=1.416943519 podStartE2EDuration="1.416943519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:29:24.415828712 +0000 UTC m=+387.856365971" watchObservedRunningTime="2026-11-03 22:29:24.416943519 +0000 UTC m=+387.857480777"
Nov 03 22:29:25.302990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:25.302960    5647 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:29:25.303399 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:25.303020    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf" podUID="2ff0e28c-db6e-4176-b124-a79a6f78ba51" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:29:28.303463 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.303429    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf"
Nov 03 22:29:28.500787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.500754    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"]
Nov 03 22:29:28.501029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.500985    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg" podUID="e810ddd3-8e80-499d-a409-12a8d143dc79" containerName="discovery" containerID="cri-o://fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b" gracePeriod=30
Nov 03 22:29:28.753972 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.753940    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:29:28.838125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.838047    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-kube-api-access-kkm7w\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-kube-api-access-kkm7w\") pod \"e810ddd3-8e80-499d-a409-12a8d143dc79\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") "
Nov 03 22:29:28.838281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.838139    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-kubeconfig\") pod \"e810ddd3-8e80-499d-a409-12a8d143dc79\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") "
Nov 03 22:29:28.838281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.838171    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-dns-cert\") pod \"e810ddd3-8e80-499d-a409-12a8d143dc79\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") "
Nov 03 22:29:28.838281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.838209    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-token\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-token\") pod \"e810ddd3-8e80-499d-a409-12a8d143dc79\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") "
Nov 03 22:29:28.838281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.838251    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-ca-configmap\") pod \"e810ddd3-8e80-499d-a409-12a8d143dc79\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") "
Nov 03 22:29:28.838478 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.838294    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-cacerts\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-cacerts\") pod \"e810ddd3-8e80-499d-a409-12a8d143dc79\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") "
Nov 03 22:29:28.838478 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.838347    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e810ddd3-8e80-499d-a409-12a8d143dc79-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e810ddd3-8e80-499d-a409-12a8d143dc79-local-certs\") pod \"e810ddd3-8e80-499d-a409-12a8d143dc79\" (UID: \"e810ddd3-8e80-499d-a409-12a8d143dc79\") "
Nov 03 22:29:28.838802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.838751    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-ca-configmap" pod "e810ddd3-8e80-499d-a409-12a8d143dc79" (UID: "e810ddd3-8e80-499d-a409-12a8d143dc79"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:29:28.840819 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.840785    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-token" pod "e810ddd3-8e80-499d-a409-12a8d143dc79" (UID: "e810ddd3-8e80-499d-a409-12a8d143dc79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:29:28.840934 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.840831    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-cacerts" pod "e810ddd3-8e80-499d-a409-12a8d143dc79" (UID: "e810ddd3-8e80-499d-a409-12a8d143dc79"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:29:28.841036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.841019    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-dns-cert" pod "e810ddd3-8e80-499d-a409-12a8d143dc79" (UID: "e810ddd3-8e80-499d-a409-12a8d143dc79"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:29:28.841103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.841068    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-kubeconfig" pod "e810ddd3-8e80-499d-a409-12a8d143dc79" (UID: "e810ddd3-8e80-499d-a409-12a8d143dc79"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:29:28.841171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.841148    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-kube-api-access-kkm7w" pod "e810ddd3-8e80-499d-a409-12a8d143dc79" (UID: "e810ddd3-8e80-499d-a409-12a8d143dc79"). InnerVolumeSpecName "kube-api-access-kkm7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:29:28.841171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.841155    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e810ddd3-8e80-499d-a409-12a8d143dc79-local-certs" pod "e810ddd3-8e80-499d-a409-12a8d143dc79" (UID: "e810ddd3-8e80-499d-a409-12a8d143dc79"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:29:28.928986 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.928951    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-66wzs"]
Nov 03 22:29:28.929325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.929313    5647 cpu_manager.go:414] "containerMap: removing container" podUID="e810ddd3-8e80-499d-a409-12a8d143dc79" containerName="discovery"
Nov 03 22:29:28.929362 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.929330    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e810ddd3-8e80-499d-a409-12a8d143dc79" containerName="discovery"
Nov 03 22:29:28.929416 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.929405    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="e810ddd3-8e80-499d-a409-12a8d143dc79" containerName="discovery"
Nov 03 22:29:28.932480 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.932463    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:28.939183 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.939155    5647 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-dns-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:29:28.939183 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.939183    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:29:28.939391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.939198    5647 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-csr-ca-configmap\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:29:28.939391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.939211    5647 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-cacerts\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:29:28.939391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.939224    5647 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e810ddd3-8e80-499d-a409-12a8d143dc79-local-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:29:28.939391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.939235    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkm7w\" (UniqueName: \"kubernetes.io/projected/e810ddd3-8e80-499d-a409-12a8d143dc79-kube-api-access-kkm7w\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:29:28.939391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.939247    5647 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e810ddd3-8e80-499d-a409-12a8d143dc79-istio-kubeconfig\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:29:28.985810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.985774    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 03 22:29:28.985810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.985774    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-b66b5\""
Nov 03 22:29:28.985810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.985805    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 03 22:29:28.986038 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.985831    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 03 22:29:28.989739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:28.989710    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-66wzs"]
Nov 03 22:29:29.040452 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.040417    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4pgr\" (UniqueName: \"kubernetes.io/projected/bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da-kube-api-access-d4pgr\") pod \"model-serving-api-75bd89fdfc-66wzs\" (UID: \"bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da\") " pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:29.040699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.040459    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da-tls-certs\") pod \"model-serving-api-75bd89fdfc-66wzs\" (UID: \"bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da\") " pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:29.042372 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.042349    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-xmkpx"]
Nov 03 22:29:29.045780 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.045760    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.098079 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.098002    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 03 22:29:29.098079 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.098043    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 03 22:29:29.098079 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.098074    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-nmzr2\""
Nov 03 22:29:29.103042 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.103019    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-xmkpx"]
Nov 03 22:29:29.141853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.141825    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46hrw\" (UniqueName: \"kubernetes.io/projected/60dcf158-4b80-418b-98bb-53e5f3b0a179-kube-api-access-46hrw\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.141991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.141864    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dcf158-4b80-418b-98bb-53e5f3b0a179-metrics-certs\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.141991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.141972    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4pgr\" (UniqueName: \"kubernetes.io/projected/bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da-kube-api-access-d4pgr\") pod \"model-serving-api-75bd89fdfc-66wzs\" (UID: \"bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da\") " pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:29.142071 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.142002    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da-tls-certs\") pod \"model-serving-api-75bd89fdfc-66wzs\" (UID: \"bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da\") " pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:29.142071 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.142019    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60dcf158-4b80-418b-98bb-53e5f3b0a179-cert\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.144575 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.144552    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da-tls-certs\") pod \"model-serving-api-75bd89fdfc-66wzs\" (UID: \"bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da\") " pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:29.210578 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.210550    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4pgr\" (UniqueName: \"kubernetes.io/projected/bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da-kube-api-access-d4pgr\") pod \"model-serving-api-75bd89fdfc-66wzs\" (UID: \"bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da\") " pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:29.242794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.242761    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60dcf158-4b80-418b-98bb-53e5f3b0a179-cert\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.242907 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.242808    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46hrw\" (UniqueName: \"kubernetes.io/projected/60dcf158-4b80-418b-98bb-53e5f3b0a179-kube-api-access-46hrw\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.242907 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.242845    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dcf158-4b80-418b-98bb-53e5f3b0a179-metrics-certs\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.245000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.244977    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:29.245242 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.245224    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60dcf158-4b80-418b-98bb-53e5f3b0a179-cert\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.245313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.245296    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dcf158-4b80-418b-98bb-53e5f3b0a179-metrics-certs\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.307614 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.307581    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46hrw\" (UniqueName: \"kubernetes.io/projected/60dcf158-4b80-418b-98bb-53e5f3b0a179-kube-api-access-46hrw\") pod \"odh-model-controller-769875db56-xmkpx\" (UID: \"60dcf158-4b80-418b-98bb-53e5f3b0a179\") " pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.320485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.320455    5647 generic.go:356] "Generic (PLEG): container finished" podID="e810ddd3-8e80-499d-a409-12a8d143dc79" containerID="fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b" exitCode=0
Nov 03 22:29:29.320652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.320519    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"
Nov 03 22:29:29.320652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.320540    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg" event={"ID":"e810ddd3-8e80-499d-a409-12a8d143dc79","Type":"ContainerDied","Data":"fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b"}
Nov 03 22:29:29.320652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.320593    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg" event={"ID":"e810ddd3-8e80-499d-a409-12a8d143dc79","Type":"ContainerDied","Data":"d92480da5129144f0f9e3937365667a9d700a0f92999dfa0b8440563fe9df8dd"}
Nov 03 22:29:29.320652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.320614    5647 scope.go:122] "RemoveContainer" containerID="fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b"
Nov 03 22:29:29.330795 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.330717    5647 scope.go:122] "RemoveContainer" containerID="fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b"
Nov 03 22:29:29.331083 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:29:29.331053    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b\": container with ID starting with fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b not found: ID does not exist" containerID="fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b"
Nov 03 22:29:29.331157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.331087    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b"} err="failed to get container status \"fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b\": rpc error: code = NotFound desc = could not find container \"fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b\": container with ID starting with fe6226cb318090a1eb5314879362442aecc67801daea7e51c01769884351c62b not found: ID does not exist"
Nov 03 22:29:29.358015 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.357950    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:29.520868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.520836    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"]
Nov 03 22:29:29.521789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.521765    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg"]
Nov 03 22:29:29.632441 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.632358    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-66wzs"]
Nov 03 22:29:29.635571 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:29:29.635540    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf9054c_8dea_49d6_a7e2_c9c6fd02c3da.slice/crio-cfa5053c2b6092168e1fafa3d9bec39ad0777a4e71c08a7fb3ec19331b2956c0 WatchSource:0}: Error finding container cfa5053c2b6092168e1fafa3d9bec39ad0777a4e71c08a7fb3ec19331b2956c0: Status 404 returned error can't find the container with id cfa5053c2b6092168e1fafa3d9bec39ad0777a4e71c08a7fb3ec19331b2956c0
Nov 03 22:29:29.729855 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:29.729703    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-xmkpx"]
Nov 03 22:29:29.732534 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:29:29.732505    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60dcf158_4b80_418b_98bb_53e5f3b0a179.slice/crio-adbffbd7cc95d36bd1d9f141c9f0218cf96edd6a9aea49648ffbc3d841e4be6e WatchSource:0}: Error finding container adbffbd7cc95d36bd1d9f141c9f0218cf96edd6a9aea49648ffbc3d841e4be6e: Status 404 returned error can't find the container with id adbffbd7cc95d36bd1d9f141c9f0218cf96edd6a9aea49648ffbc3d841e4be6e
Nov 03 22:29:30.331868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:30.331825    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-xmkpx" event={"ID":"60dcf158-4b80-418b-98bb-53e5f3b0a179","Type":"ContainerStarted","Data":"adbffbd7cc95d36bd1d9f141c9f0218cf96edd6a9aea49648ffbc3d841e4be6e"}
Nov 03 22:29:30.333285 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:30.333258    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-66wzs" event={"ID":"bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da","Type":"ContainerStarted","Data":"cfa5053c2b6092168e1fafa3d9bec39ad0777a4e71c08a7fb3ec19331b2956c0"}
Nov 03 22:29:30.698078 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:30.697983    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e810ddd3-8e80-499d-a409-12a8d143dc79" path="/var/lib/kubelet/pods/e810ddd3-8e80-499d-a409-12a8d143dc79/volumes"
Nov 03 22:29:32.351101 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:32.351000    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-66wzs" event={"ID":"bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da","Type":"ContainerStarted","Data":"c4d78eae7b5d68ea70975f50ccccd1d7726d97f7c404e68b6cffed478c8517ec"}
Nov 03 22:29:32.351593 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:32.351164    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:32.467212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:32.467156    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-66wzs" podStartSLOduration=2.11425863 podStartE2EDuration="4.467141431s" totalImagesPullingTime="2.352882801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:29:32.466515842 +0000 UTC m=+395.907053100" watchObservedRunningTime="2026-11-03 22:29:32.467141431 +0000 UTC m=+395.907678687"
Nov 03 22:29:33.356711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:33.356597    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-xmkpx" event={"ID":"60dcf158-4b80-418b-98bb-53e5f3b0a179","Type":"ContainerStarted","Data":"8f153b524a868a6ca0fe1974fe1f03f173b31ca8d1a5a982581d6f0f993c7527"}
Nov 03 22:29:33.473067 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:33.473001    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-xmkpx" podStartSLOduration=1.204929052 podStartE2EDuration="4.472982758s" totalImagesPullingTime="3.268053706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:29:33.472305924 +0000 UTC m=+396.912843184" watchObservedRunningTime="2026-11-03 22:29:33.472982758 +0000 UTC m=+396.913520017"
Nov 03 22:29:34.362027 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:34.361983    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:39.561455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:39.561422    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:29:43.361365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:43.361336    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-66wzs"
Nov 03 22:29:44.741105 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.741069    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"]
Nov 03 22:29:44.744347 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.744326    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.845433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.845405    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 22:29:44.845640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.845521    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 22:29:44.845640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.845533    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 03 22:29:44.845640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.845542    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 03 22:29:44.845640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.845623    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sw68g\""
Nov 03 22:29:44.846900 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.846882    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"]
Nov 03 22:29:44.885513 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.885482    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-metrics-certs\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.885654 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.885553    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmctt\" (UniqueName: \"kubernetes.io/projected/a14e804a-62c7-468b-b465-ec62bff41bbb-kube-api-access-kmctt\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.885654 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.885642    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-maas-api-tls\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.885766 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.885696    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14e804a-62c7-468b-b465-ec62bff41bbb-openshift-service-ca\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.986531 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.986494    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmctt\" (UniqueName: \"kubernetes.io/projected/a14e804a-62c7-468b-b465-ec62bff41bbb-kube-api-access-kmctt\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.986737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.986543    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-maas-api-tls\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.986737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.986560    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14e804a-62c7-468b-b465-ec62bff41bbb-openshift-service-ca\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.986737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.986588    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-metrics-certs\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.987329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.987300    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14e804a-62c7-468b-b465-ec62bff41bbb-openshift-service-ca\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.989044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.989025    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-maas-api-tls\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:44.989170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:44.989152    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-metrics-certs\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:45.047211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:45.047137    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmctt\" (UniqueName: \"kubernetes.io/projected/a14e804a-62c7-468b-b465-ec62bff41bbb-kube-api-access-kmctt\") pod \"maas-api-6ddf589cff-94hxg\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") " pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:45.056090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:45.056062    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:45.368607 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:45.368525    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-xmkpx"
Nov 03 22:29:45.428186 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:45.428159    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"]
Nov 03 22:29:45.431604 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:29:45.431572    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14e804a_62c7_468b_b465_ec62bff41bbb.slice/crio-98e9f3eb9ca5529d00eb2c32b53bef2dd7ea98a38b50c0a72d0cd2ad54305af5 WatchSource:0}: Error finding container 98e9f3eb9ca5529d00eb2c32b53bef2dd7ea98a38b50c0a72d0cd2ad54305af5: Status 404 returned error can't find the container with id 98e9f3eb9ca5529d00eb2c32b53bef2dd7ea98a38b50c0a72d0cd2ad54305af5
Nov 03 22:29:46.041358 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:29:46.041324    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:29:46.417495 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.417401    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg" event={"ID":"a14e804a-62c7-468b-b465-ec62bff41bbb","Type":"ContainerStarted","Data":"98e9f3eb9ca5529d00eb2c32b53bef2dd7ea98a38b50c0a72d0cd2ad54305af5"}
Nov 03 22:29:46.819776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.819734    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"]
Nov 03 22:29:46.823370 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.823346    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.875184 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.875158    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-89xv7\""
Nov 03 22:29:46.905332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905296    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.905495 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905348    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38825124-f803-4234-ad30-349fb215b0b1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.905495 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905440    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38825124-f803-4234-ad30-349fb215b0b1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.905612 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905500    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.905683 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905637    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.905749 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905663    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.905798 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905764    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.905848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905802    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.905848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.905823    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpgw9\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-kube-api-access-fpgw9\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:46.926178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:46.926102    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"]
Nov 03 22:29:47.006528 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.006486    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.006719 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.006565    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.006719 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.006590    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.006719 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.006621    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.006719 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.006646    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.006996 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.006962    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpgw9\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-kube-api-access-fpgw9\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.007057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.007036    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.007057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.007043    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.007159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.007074    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38825124-f803-4234-ad30-349fb215b0b1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.007159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.007094    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.007159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.007102    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38825124-f803-4234-ad30-349fb215b0b1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.007417 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.007168    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.007471 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.007456    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.007864 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.007816    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38825124-f803-4234-ad30-349fb215b0b1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.009560 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.009536    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.009737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.009714    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38825124-f803-4234-ad30-349fb215b0b1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.074255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.074176    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.074640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.074423    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpgw9\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-kube-api-access-fpgw9\") pod \"maas-default-gateway-openshift-default-7f5667f86c-k2njq\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.142611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.142575    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:47.577629 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:47.577596    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"]
Nov 03 22:29:47.582764 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:29:47.582730    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38825124_f803_4234_ad30_349fb215b0b1.slice/crio-594425c6bf53b8cf1d575e893488493511e8eb79d44420abbca4b622f6f5b074 WatchSource:0}: Error finding container 594425c6bf53b8cf1d575e893488493511e8eb79d44420abbca4b622f6f5b074: Status 404 returned error can't find the container with id 594425c6bf53b8cf1d575e893488493511e8eb79d44420abbca4b622f6f5b074
Nov 03 22:29:48.428696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:48.428644    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg" event={"ID":"a14e804a-62c7-468b-b465-ec62bff41bbb","Type":"ContainerStarted","Data":"a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112"}
Nov 03 22:29:48.429181 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:48.428738    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:48.429978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:48.429952    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" event={"ID":"38825124-f803-4234-ad30-349fb215b0b1","Type":"ContainerStarted","Data":"594425c6bf53b8cf1d575e893488493511e8eb79d44420abbca4b622f6f5b074"}
Nov 03 22:29:48.548701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:48.548579    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg" podStartSLOduration=1.761056363 podStartE2EDuration="4.548564102s" totalImagesPullingTime="2.787507739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:29:48.547576571 +0000 UTC m=+411.988113843" watchObservedRunningTime="2026-11-03 22:29:48.548564102 +0000 UTC m=+411.989101390"
Nov 03 22:29:51.250736 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:51.250705    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:29:51.251017 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:51.250769    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:29:51.251017 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:51.250795    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:29:51.446165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:51.446127    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" event={"ID":"38825124-f803-4234-ad30-349fb215b0b1","Type":"ContainerStarted","Data":"c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf"}
Nov 03 22:29:51.614449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:51.614327    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podStartSLOduration=1.9496331599999999 podStartE2EDuration="5.614311214s" totalImagesPullingTime="3.664678054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:29:51.61427062 +0000 UTC m=+415.054807878" watchObservedRunningTime="2026-11-03 22:29:51.614311214 +0000 UTC m=+415.054848474"
Nov 03 22:29:52.143403 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:52.143365    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:52.144882 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:52.144856    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 03 22:29:52.145001 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:52.144904    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 03 22:29:53.143059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:53.143025    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 03 22:29:53.143441 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:53.143077    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 03 22:29:54.143480 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:54.143438    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 03 22:29:54.143993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:54.143510    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 03 22:29:54.440930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:54.440863    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 22:29:55.142928 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:55.142892    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 03 22:29:55.143106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:55.142946    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 03 22:29:56.143045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:56.143011    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 03 22:29:56.143511 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:56.143084    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 03 22:29:57.143780 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:57.143683    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Nov 03 22:29:57.143780 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:57.143741    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Nov 03 22:29:58.146958 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:58.146928    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:58.147348 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:58.147183    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:29:58.147984 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:29:58.147969    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:30:16.041358 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:30:16.041317    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:30:46.041582 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:30:46.041543    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:31:16.040848 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:31:16.040794    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:31:42.313342 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.313304    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"]
Nov 03 22:31:42.315629 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.315613    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.367600 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.367570    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 03 22:31:42.369011 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.368989    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p26fw\" (UniqueName: \"kubernetes.io/projected/1b27706b-6628-4cc7-812e-e22541de8064-kube-api-access-p26fw\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.369100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.369023    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.369100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.369051    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.369171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.369105    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.369171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.369136    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b27706b-6628-4cc7-812e-e22541de8064-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.369310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.369297    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.416807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.416768    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-h7bmr\""
Nov 03 22:31:42.416807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.416800    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 03 22:31:42.417014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.416784    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 03 22:31:42.418116 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.418096    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"]
Nov 03 22:31:42.470617 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.470583    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p26fw\" (UniqueName: \"kubernetes.io/projected/1b27706b-6628-4cc7-812e-e22541de8064-kube-api-access-p26fw\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.470617 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.470624    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.470847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.470652    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.470883 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.470837    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.470883 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.470870    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b27706b-6628-4cc7-812e-e22541de8064-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.470947 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.470921    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.471312 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.471291    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.471353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.471319    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.471353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.471345    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.473433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.473406    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b27706b-6628-4cc7-812e-e22541de8064-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.473584 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.473567    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b27706b-6628-4cc7-812e-e22541de8064-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.532720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.532683    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p26fw\" (UniqueName: \"kubernetes.io/projected/1b27706b-6628-4cc7-812e-e22541de8064-kube-api-access-p26fw\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj\" (UID: \"1b27706b-6628-4cc7-812e-e22541de8064\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.626892 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.626780    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:42.997254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:42.997231    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"]
Nov 03 22:31:43.000004 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:31:42.999972    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b27706b_6628_4cc7_812e_e22541de8064.slice/crio-9a66eef5b53901b4c2f6cc6d362dd279086bc12fd532a96039b178df3f51cbcc WatchSource:0}: Error finding container 9a66eef5b53901b4c2f6cc6d362dd279086bc12fd532a96039b178df3f51cbcc: Status 404 returned error can't find the container with id 9a66eef5b53901b4c2f6cc6d362dd279086bc12fd532a96039b178df3f51cbcc
Nov 03 22:31:43.937184 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:43.937141    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj" event={"ID":"1b27706b-6628-4cc7-812e-e22541de8064","Type":"ContainerStarted","Data":"9a66eef5b53901b4c2f6cc6d362dd279086bc12fd532a96039b178df3f51cbcc"}
Nov 03 22:31:45.791994 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:45.791530    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:31:45.949250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:45.949210    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj" event={"ID":"1b27706b-6628-4cc7-812e-e22541de8064","Type":"ContainerStarted","Data":"db865f150f465e6207cd52ef6c3c33012a0b8f054bb6b5781b87207f6049acd7"}
Nov 03 22:31:46.050097 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:31:46.049942    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:31:46.066851 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:46.066801    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj" podStartSLOduration=2.136425595 podStartE2EDuration="4.066784368s" totalImagesPullingTime="1.930358773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:31:46.065042588 +0000 UTC m=+529.505579869" watchObservedRunningTime="2026-11-03 22:31:46.066784368 +0000 UTC m=+529.507321627"
Nov 03 22:31:46.627820 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:46.627787    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:52.627273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:52.627230    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:52.639907 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:52.639885    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:52.992771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:52.992738    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj"
Nov 03 22:31:55.252283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:55.252251    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:31:58.507191 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.507155    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"]
Nov 03 22:31:58.510252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.510230    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.562101 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.562073    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Nov 03 22:31:58.569229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.569201    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"]
Nov 03 22:31:58.626443 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.626413    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.626640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.626478    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.626640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.626598    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l84n\" (UniqueName: \"kubernetes.io/projected/f01a335e-fe1d-40cd-87ca-8611d04c3404-kube-api-access-5l84n\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.626640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.626635    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.626839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.626652    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f01a335e-fe1d-40cd-87ca-8611d04c3404-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.626839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.626721    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.727735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.727602    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l84n\" (UniqueName: \"kubernetes.io/projected/f01a335e-fe1d-40cd-87ca-8611d04c3404-kube-api-access-5l84n\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.727735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.727732    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.727990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.727762    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f01a335e-fe1d-40cd-87ca-8611d04c3404-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.727990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.727796    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.727990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.727860    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.727990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.727933    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.728200 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.728172    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.728571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.728524    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.729161 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.728603    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.730711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.730659    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f01a335e-fe1d-40cd-87ca-8611d04c3404-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.730955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.730936    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f01a335e-fe1d-40cd-87ca-8611d04c3404-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.794380 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.794294    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l84n\" (UniqueName: \"kubernetes.io/projected/f01a335e-fe1d-40cd-87ca-8611d04c3404-kube-api-access-5l84n\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-c7zcs\" (UID: \"f01a335e-fe1d-40cd-87ca-8611d04c3404\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:58.821039 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:58.821002    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:31:59.201359 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:31:59.201277    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"]
Nov 03 22:31:59.205166 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:31:59.205132    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf01a335e_fe1d_40cd_87ca_8611d04c3404.slice/crio-caaddeaca069506a655136130aab8e78104ac8657156fddade6e7c2de29202cf WatchSource:0}: Error finding container caaddeaca069506a655136130aab8e78104ac8657156fddade6e7c2de29202cf: Status 404 returned error can't find the container with id caaddeaca069506a655136130aab8e78104ac8657156fddade6e7c2de29202cf
Nov 03 22:32:00.013408 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:00.013368    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs" event={"ID":"f01a335e-fe1d-40cd-87ca-8611d04c3404","Type":"ContainerStarted","Data":"7cb8ff11c631181b0e825e6e5683985d65b458d26f43eec06a7a6509e32d4c20"}
Nov 03 22:32:00.013846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:00.013472    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs" event={"ID":"f01a335e-fe1d-40cd-87ca-8611d04c3404","Type":"ContainerStarted","Data":"caaddeaca069506a655136130aab8e78104ac8657156fddade6e7c2de29202cf"}
Nov 03 22:32:00.132780 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:00.132637    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs" podStartSLOduration=1.5370048280000002 podStartE2EDuration="2.13261804s" totalImagesPullingTime="595.613212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:31:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:00.131325583 +0000 UTC m=+543.571862843" watchObservedRunningTime="2026-11-03 22:32:00.13261804 +0000 UTC m=+543.573155299"
Nov 03 22:32:00.821934 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:00.821896    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:32:02.816890 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.816853    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"]
Nov 03 22:32:02.819449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.819428    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.924002 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.923975    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"]
Nov 03 22:32:02.971839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.971798    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.971839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.971842    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e4508fb-4d66-4503-826a-155a08bcc9df-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.972050 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.971870    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.972050 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.972016    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.972116 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.972051    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46h5n\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-kube-api-access-46h5n\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.972154 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.972123    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.972252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.972234    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.972286 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.972270    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:02.972372 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:02.972360    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.073795 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.073708    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.073795 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.073761    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.073830    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.073864    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.073888    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e4508fb-4d66-4503-826a-155a08bcc9df-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.073916    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.073966    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.074286    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074400 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.074330    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074400 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.074370    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46h5n\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-kube-api-access-46h5n\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.074415    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.074442    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074733 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.074707    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.074817 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.074776    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e4508fb-4d66-4503-826a-155a08bcc9df-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.076542 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.076508    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.076646 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.076600    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.134656 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.134625    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.134851 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.134830    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46h5n\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-kube-api-access-46h5n\") pod \"maas-default-gateway-openshift-default-7f5667f86c-kr95p\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.431788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.431697    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:03.856383 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.856351    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"]
Nov 03 22:32:03.858880 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:32:03.858855    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4508fb_4d66_4503_826a_155a08bcc9df.slice/crio-787b8509048d0c4ef3812dea825d48b8da31fcbff097759a00c9115a98023df0 WatchSource:0}: Error finding container 787b8509048d0c4ef3812dea825d48b8da31fcbff097759a00c9115a98023df0: Status 404 returned error can't find the container with id 787b8509048d0c4ef3812dea825d48b8da31fcbff097759a00c9115a98023df0
Nov 03 22:32:03.861058 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.861027    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:32:03.861137 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.861099    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:32:03.861137 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:03.861128    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:32:04.033447 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:04.033380    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" event={"ID":"6e4508fb-4d66-4503-826a-155a08bcc9df","Type":"ContainerStarted","Data":"a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20"}
Nov 03 22:32:04.033447 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:04.033444    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" event={"ID":"6e4508fb-4d66-4503-826a-155a08bcc9df","Type":"ContainerStarted","Data":"787b8509048d0c4ef3812dea825d48b8da31fcbff097759a00c9115a98023df0"}
Nov 03 22:32:04.153972 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:04.153863    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podStartSLOduration=2.15384429 podStartE2EDuration="2.15384429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:04.152766824 +0000 UTC m=+547.593304072" watchObservedRunningTime="2026-11-03 22:32:04.15384429 +0000 UTC m=+547.594381548"
Nov 03 22:32:04.432250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:04.432153    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:04.433686 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:04.433642    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 03 22:32:04.433792 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:04.433713    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 03 22:32:04.615482 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:04.615450    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:32:05.247542 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.247502    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"]
Nov 03 22:32:05.251681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.251649    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.305505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.305472    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 03 22:32:05.356035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.356000    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"]
Nov 03 22:32:05.409449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.409412    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5xh9\" (UniqueName: \"kubernetes.io/projected/90a88463-27e7-4a04-86d7-ef9ba10d9a08-kube-api-access-g5xh9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.409699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.409470    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.409699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.409502    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.409699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.409538    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.409699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.409562    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.409699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.409592    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90a88463-27e7-4a04-86d7-ef9ba10d9a08-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.432306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.432256    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 03 22:32:05.432477 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.432329    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 03 22:32:05.510267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.510159    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90a88463-27e7-4a04-86d7-ef9ba10d9a08-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.510267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.510264    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xh9\" (UniqueName: \"kubernetes.io/projected/90a88463-27e7-4a04-86d7-ef9ba10d9a08-kube-api-access-g5xh9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.510505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.510319    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.510505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.510347    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.510505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.510384    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.510505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.510411    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.511061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.510939    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.511314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.511288    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.511532 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.511505    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.513457 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.513434    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90a88463-27e7-4a04-86d7-ef9ba10d9a08-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.513833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.513816    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90a88463-27e7-4a04-86d7-ef9ba10d9a08-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.588692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.582614    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5xh9\" (UniqueName: \"kubernetes.io/projected/90a88463-27e7-4a04-86d7-ef9ba10d9a08-kube-api-access-g5xh9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q\" (UID: \"90a88463-27e7-4a04-86d7-ef9ba10d9a08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:05.866110 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:05.866017    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:06.281144 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:06.281106    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"]
Nov 03 22:32:06.295985 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:32:06.295838    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90a88463_27e7_4a04_86d7_ef9ba10d9a08.slice/crio-aa5a35c1746e5c810f2bf7af58f0aa6c06b6aaec273c70e804bbab6365bf0991 WatchSource:0}: Error finding container aa5a35c1746e5c810f2bf7af58f0aa6c06b6aaec273c70e804bbab6365bf0991: Status 404 returned error can't find the container with id aa5a35c1746e5c810f2bf7af58f0aa6c06b6aaec273c70e804bbab6365bf0991
Nov 03 22:32:06.432695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:06.432631    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 03 22:32:06.432875 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:06.432736    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 03 22:32:07.051135 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:07.051102    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q" event={"ID":"90a88463-27e7-4a04-86d7-ef9ba10d9a08","Type":"ContainerStarted","Data":"018bbb8d3fe17a5813750c96e6e4fbb0e4816cd7e0ee64611a120c907388712e"}
Nov 03 22:32:07.051320 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:07.051154    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q" event={"ID":"90a88463-27e7-4a04-86d7-ef9ba10d9a08","Type":"ContainerStarted","Data":"aa5a35c1746e5c810f2bf7af58f0aa6c06b6aaec273c70e804bbab6365bf0991"}
Nov 03 22:32:07.209494 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:07.209422    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q" podStartSLOduration=1.572929722 podStartE2EDuration="2.209403635s" totalImagesPullingTime="636.473913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:07.208782021 +0000 UTC m=+550.649319281" watchObservedRunningTime="2026-11-03 22:32:07.209403635 +0000 UTC m=+550.649940894"
Nov 03 22:32:07.432324 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:07.432280    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 03 22:32:07.432733 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:07.432339    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 03 22:32:07.866367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:07.866336    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:08.433015 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:08.432972    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 03 22:32:08.433407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:08.433029    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 03 22:32:08.821280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:08.821237    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:32:08.833843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:08.833818    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:32:09.098115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:09.098024    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-c7zcs"
Nov 03 22:32:09.433060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:09.432966    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 03 22:32:09.433060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:09.433026    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 03 22:32:11.473833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:11.473785    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:32:11.474376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:11.473847    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:32:12.113602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:12.113568    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:12.113959 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:12.113935    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:12.114699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:12.114655    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:32:15.724252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.724221    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"]
Nov 03 22:32:15.727603 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.727586    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.816208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.816154    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.816208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.816207    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.816445 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.816303    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.816445 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.816346    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.816558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.816454    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.816624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.816602    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8wwx\" (UniqueName: \"kubernetes.io/projected/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-kube-api-access-l8wwx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.828804 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.828772    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 03 22:32:15.830285 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.830264    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"]
Nov 03 22:32:15.869878 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.869844    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:15.885540 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.885512    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:15.918129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918096    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.918302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918184    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8wwx\" (UniqueName: \"kubernetes.io/projected/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-kube-api-access-l8wwx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.918302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918205    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.918302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918229    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.918302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918259    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.918513 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918311    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.918738 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918704    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.918835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918711    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.918835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.918780    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.920635 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.920605    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.920978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.920955    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:15.979946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:15.979868    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8wwx\" (UniqueName: \"kubernetes.io/projected/4fd7ac8b-8570-444e-ad0b-66ed03b79fa2-kube-api-access-l8wwx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75\" (UID: \"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:16.040868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:16.040840    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:16.042514 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:32:16.042477    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:32:16.128470 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:16.128444    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q"
Nov 03 22:32:16.438803 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:16.438769    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"]
Nov 03 22:32:16.442846 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:32:16.442812    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd7ac8b_8570_444e_ad0b_66ed03b79fa2.slice/crio-f5a4badc824185f0c95c90dafb6d34e4e58cbe9f8ad1648f1e43e605740d6e4d WatchSource:0}: Error finding container f5a4badc824185f0c95c90dafb6d34e4e58cbe9f8ad1648f1e43e605740d6e4d: Status 404 returned error can't find the container with id f5a4badc824185f0c95c90dafb6d34e4e58cbe9f8ad1648f1e43e605740d6e4d
Nov 03 22:32:17.116435 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:17.116402    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75" event={"ID":"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2","Type":"ContainerStarted","Data":"f5a4badc824185f0c95c90dafb6d34e4e58cbe9f8ad1648f1e43e605740d6e4d"}
Nov 03 22:32:17.938718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:17.938657    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"]
Nov 03 22:32:17.943113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:17.943086    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.043293 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.043236    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.043588 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.043514    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.043818 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.043725    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.043934 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.043851    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e56eae5f-6e03-493d-a288-c29f37f06b3a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.044014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.043997    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.044072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.044034    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.044172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.044147    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.044227 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.044197    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.044335 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.044309    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggkg\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-kube-api-access-fggkg\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.046192 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.046167    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"]
Nov 03 22:32:18.123564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.123522    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75" event={"ID":"4fd7ac8b-8570-444e-ad0b-66ed03b79fa2","Type":"ContainerStarted","Data":"cd4900d65aa673e10c82885b01f961587e4dc42bf893b63e7121c597e70911cf"}
Nov 03 22:32:18.145181 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145145    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggkg\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-kube-api-access-fggkg\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145189    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145209    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145262    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145291    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e56eae5f-6e03-493d-a288-c29f37f06b3a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145342    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145370    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145416    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145457    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145776    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.145968 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145824    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.146035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.145894    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.146035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.146020    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.146203 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.146173    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e56eae5f-6e03-493d-a288-c29f37f06b3a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.147686 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.147653    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.148007 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.147985    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.206097 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.206059    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.206274 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.206126    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggkg\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-kube-api-access-fggkg\") pod \"maas-default-gateway-openshift-default-7f5667f86c-xtmcj\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.241130 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.241075    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75" podStartSLOduration=2.6397698910000003 podStartE2EDuration="3.241058025s" totalImagesPullingTime="601.288134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:18.240631819 +0000 UTC m=+561.681169078" watchObservedRunningTime="2026-11-03 22:32:18.241058025 +0000 UTC m=+561.681595285"
Nov 03 22:32:18.257414 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.257381    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:18.684350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.684278    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"]
Nov 03 22:32:18.687541 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:32:18.687504    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode56eae5f_6e03_493d_a288_c29f37f06b3a.slice/crio-35266ea9e51626532e9ea865c85f7c18f310d9803c1356877871f3b6baff60f4 WatchSource:0}: Error finding container 35266ea9e51626532e9ea865c85f7c18f310d9803c1356877871f3b6baff60f4: Status 404 returned error can't find the container with id 35266ea9e51626532e9ea865c85f7c18f310d9803c1356877871f3b6baff60f4
Nov 03 22:32:18.689865 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.689834    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:32:18.689947 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.689901    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:32:18.689947 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:18.689928    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:32:19.042035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:19.041997    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:19.134700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:19.133873    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" event={"ID":"e56eae5f-6e03-493d-a288-c29f37f06b3a","Type":"ContainerStarted","Data":"d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f"}
Nov 03 22:32:19.134700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:19.133937    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" event={"ID":"e56eae5f-6e03-493d-a288-c29f37f06b3a","Type":"ContainerStarted","Data":"35266ea9e51626532e9ea865c85f7c18f310d9803c1356877871f3b6baff60f4"}
Nov 03 22:32:19.257783 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:19.257743    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:19.259456 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:19.259422    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 03 22:32:19.259614 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:19.259486    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 03 22:32:19.260044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:19.259995    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podStartSLOduration=2.259982482 podStartE2EDuration="2.259982482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:19.259203516 +0000 UTC m=+562.699740787" watchObservedRunningTime="2026-11-03 22:32:19.259982482 +0000 UTC m=+562.700519742"
Nov 03 22:32:20.258475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:20.258432    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 03 22:32:20.258997 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:20.258494    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 03 22:32:20.597826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:20.597725    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:32:21.258603 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:21.258568    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 03 22:32:21.258998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:21.258635    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 03 22:32:22.258267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:22.258218    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 03 22:32:22.258533 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:22.258283    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 03 22:32:23.258363 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:23.258316    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 03 22:32:23.258796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:23.258761    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 03 22:32:24.258779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:24.258747    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 03 22:32:24.259262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:24.258802    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 03 22:32:25.258520 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:25.258465    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 03 22:32:25.258767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:25.258533    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 03 22:32:26.042087 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:26.042048    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:26.055611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:26.055579    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:26.217576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:26.217542    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75"
Nov 03 22:32:27.258962 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:27.258912    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:32:27.259491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:27.258988    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:32:28.300926 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:28.300876    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:32:28.301385 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:28.300969    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:32:28.712900 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:28.712867    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:28.713202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:28.713165    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:28.713896 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:28.713878    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:32:43.843027 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:43.842991    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-29zcf"]
Nov 03 22:32:43.846578 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:43.846559    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:43.899971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:43.899934    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-k6m24\""
Nov 03 22:32:43.949507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:43.949472    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 03 22:32:43.949801 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:43.949776    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 03 22:32:43.950980 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:43.950961    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-29zcf"]
Nov 03 22:32:43.961412 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:43.961385    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlb5q\" (UniqueName: \"kubernetes.io/projected/bd7703d5-d970-49c1-8865-87fc8b19c2ef-kube-api-access-nlb5q\") pod \"ai-gateway-controller-55cd769764-29zcf\" (UID: \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\") " pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:43.961557 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:43.961453    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7703d5-d970-49c1-8865-87fc8b19c2ef-metrics-certs\") pod \"ai-gateway-controller-55cd769764-29zcf\" (UID: \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\") " pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:44.062819 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:44.062782    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlb5q\" (UniqueName: \"kubernetes.io/projected/bd7703d5-d970-49c1-8865-87fc8b19c2ef-kube-api-access-nlb5q\") pod \"ai-gateway-controller-55cd769764-29zcf\" (UID: \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\") " pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:44.063019 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:44.062856    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7703d5-d970-49c1-8865-87fc8b19c2ef-metrics-certs\") pod \"ai-gateway-controller-55cd769764-29zcf\" (UID: \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\") " pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:44.065394 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:44.065362    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7703d5-d970-49c1-8865-87fc8b19c2ef-metrics-certs\") pod \"ai-gateway-controller-55cd769764-29zcf\" (UID: \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\") " pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:44.123615 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:44.123539    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlb5q\" (UniqueName: \"kubernetes.io/projected/bd7703d5-d970-49c1-8865-87fc8b19c2ef-kube-api-access-nlb5q\") pod \"ai-gateway-controller-55cd769764-29zcf\" (UID: \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\") " pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:44.159789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:44.159750    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:44.535839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:44.535754    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-29zcf"]
Nov 03 22:32:44.538226 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:32:44.538198    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7703d5_d970_49c1_8865_87fc8b19c2ef.slice/crio-49093a288da0422321f2324c2e7c6a29ba8714893b74d93ce94d32da4370928d WatchSource:0}: Error finding container 49093a288da0422321f2324c2e7c6a29ba8714893b74d93ce94d32da4370928d: Status 404 returned error can't find the container with id 49093a288da0422321f2324c2e7c6a29ba8714893b74d93ce94d32da4370928d
Nov 03 22:32:45.302711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:45.302662    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf" event={"ID":"bd7703d5-d970-49c1-8865-87fc8b19c2ef","Type":"ContainerStarted","Data":"49093a288da0422321f2324c2e7c6a29ba8714893b74d93ce94d32da4370928d"}
Nov 03 22:32:46.051304 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:32:46.051235    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:32:48.328614 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:48.328577    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf" event={"ID":"bd7703d5-d970-49c1-8865-87fc8b19c2ef","Type":"ContainerStarted","Data":"dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6"}
Nov 03 22:32:48.329097 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:48.328727    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:48.449274 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:48.449201    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf" podStartSLOduration=2.44979985 podStartE2EDuration="5.44918734s" totalImagesPullingTime="2.99938749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:48.44251395 +0000 UTC m=+591.883051209" watchObservedRunningTime="2026-11-03 22:32:48.44918734 +0000 UTC m=+591.889724598"
Nov 03 22:32:51.884699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:51.884645    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"]
Nov 03 22:32:51.888591 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:51.888575    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:51.944287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:51.944255    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Nov 03 22:32:51.953516 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:51.953489    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8k5f\" (UniqueName: \"kubernetes.io/projected/cf4f293e-6230-416c-ac10-2820e7c01188-kube-api-access-v8k5f\") pod \"payload-pre-processing-7b98bb678c-vxr7w\" (UID: \"cf4f293e-6230-416c-ac10-2820e7c01188\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:51.953652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:51.953600    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf4f293e-6230-416c-ac10-2820e7c01188-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-vxr7w\" (UID: \"cf4f293e-6230-416c-ac10-2820e7c01188\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:51.995435 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:51.995400    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"]
Nov 03 22:32:51.995619 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:51.995581    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8wz8v\""
Nov 03 22:32:52.041884 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.041842    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wqbz4"]
Nov 03 22:32:52.045479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.045454    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:52.054550 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.054517    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf4f293e-6230-416c-ac10-2820e7c01188-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-vxr7w\" (UID: \"cf4f293e-6230-416c-ac10-2820e7c01188\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:52.055482 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.055406    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf4f293e-6230-416c-ac10-2820e7c01188-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-vxr7w\" (UID: \"cf4f293e-6230-416c-ac10-2820e7c01188\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:52.055606 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.055587    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8k5f\" (UniqueName: \"kubernetes.io/projected/cf4f293e-6230-416c-ac10-2820e7c01188-kube-api-access-v8k5f\") pod \"payload-pre-processing-7b98bb678c-vxr7w\" (UID: \"cf4f293e-6230-416c-ac10-2820e7c01188\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:52.102477 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.102439    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wqbz4"]
Nov 03 22:32:52.116906 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.116879    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8k5f\" (UniqueName: \"kubernetes.io/projected/cf4f293e-6230-416c-ac10-2820e7c01188-kube-api-access-v8k5f\") pod \"payload-pre-processing-7b98bb678c-vxr7w\" (UID: \"cf4f293e-6230-416c-ac10-2820e7c01188\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:52.157230 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.157148    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pz42\" (UniqueName: \"kubernetes.io/projected/ba296c09-65ef-4276-be45-96373f18b09a-kube-api-access-7pz42\") pod \"payload-processing-7bb9489676-wqbz4\" (UID: \"ba296c09-65ef-4276-be45-96373f18b09a\") " pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:52.157230 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.157197    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba296c09-65ef-4276-be45-96373f18b09a-plugins-config-volume\") pod \"payload-processing-7bb9489676-wqbz4\" (UID: \"ba296c09-65ef-4276-be45-96373f18b09a\") " pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:52.201445 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.201406    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:52.259643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.258529    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pz42\" (UniqueName: \"kubernetes.io/projected/ba296c09-65ef-4276-be45-96373f18b09a-kube-api-access-7pz42\") pod \"payload-processing-7bb9489676-wqbz4\" (UID: \"ba296c09-65ef-4276-be45-96373f18b09a\") " pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:52.259643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.258581    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba296c09-65ef-4276-be45-96373f18b09a-plugins-config-volume\") pod \"payload-processing-7bb9489676-wqbz4\" (UID: \"ba296c09-65ef-4276-be45-96373f18b09a\") " pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:52.259643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.259353    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba296c09-65ef-4276-be45-96373f18b09a-plugins-config-volume\") pod \"payload-processing-7bb9489676-wqbz4\" (UID: \"ba296c09-65ef-4276-be45-96373f18b09a\") " pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:52.320013 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.319983    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pz42\" (UniqueName: \"kubernetes.io/projected/ba296c09-65ef-4276-be45-96373f18b09a-kube-api-access-7pz42\") pod \"payload-processing-7bb9489676-wqbz4\" (UID: \"ba296c09-65ef-4276-be45-96373f18b09a\") " pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:52.363557 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.363532    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:52.570785 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.570759    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"]
Nov 03 22:32:52.572628 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:32:52.572598    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4f293e_6230_416c_ac10_2820e7c01188.slice/crio-2b5fbd03382cdcb08909665a2d0209f3672b3155b89a69a7b841025ada3dc46a WatchSource:0}: Error finding container 2b5fbd03382cdcb08909665a2d0209f3672b3155b89a69a7b841025ada3dc46a: Status 404 returned error can't find the container with id 2b5fbd03382cdcb08909665a2d0209f3672b3155b89a69a7b841025ada3dc46a
Nov 03 22:32:52.735944 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:52.735777    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wqbz4"]
Nov 03 22:32:52.738873 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:32:52.738840    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba296c09_65ef_4276_be45_96373f18b09a.slice/crio-82890f4c8d4af536d89cb24bea04ec5c55b0101d32735aa1fb9651d7fd0992e3 WatchSource:0}: Error finding container 82890f4c8d4af536d89cb24bea04ec5c55b0101d32735aa1fb9651d7fd0992e3: Status 404 returned error can't find the container with id 82890f4c8d4af536d89cb24bea04ec5c55b0101d32735aa1fb9651d7fd0992e3
Nov 03 22:32:53.353812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:53.353780    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-wqbz4" event={"ID":"ba296c09-65ef-4276-be45-96373f18b09a","Type":"ContainerStarted","Data":"82890f4c8d4af536d89cb24bea04ec5c55b0101d32735aa1fb9651d7fd0992e3"}
Nov 03 22:32:53.354965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:53.354944    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w" event={"ID":"cf4f293e-6230-416c-ac10-2820e7c01188","Type":"ContainerStarted","Data":"2b5fbd03382cdcb08909665a2d0209f3672b3155b89a69a7b841025ada3dc46a"}
Nov 03 22:32:56.379343 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:56.379304    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-wqbz4" event={"ID":"ba296c09-65ef-4276-be45-96373f18b09a","Type":"ContainerStarted","Data":"c46ca7bf899e8e39ee45e5aa4dac68a248a6cc4fb13b184823dc8f4e62570419"}
Nov 03 22:32:56.379889 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:56.379404    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:32:56.380934 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:56.380906    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w" event={"ID":"cf4f293e-6230-416c-ac10-2820e7c01188","Type":"ContainerStarted","Data":"14a981ee4dc6b647d1ba6784e29fc272f658cc296042f087c2ff91a255add164"}
Nov 03 22:32:56.381116 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:56.380993    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:32:56.503239 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:56.503181    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-wqbz4" podStartSLOduration=2.875874825 podStartE2EDuration="5.503166616s" totalImagesPullingTime="2.627291791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:56.501300889 +0000 UTC m=+599.941838147" watchObservedRunningTime="2026-11-03 22:32:56.503166616 +0000 UTC m=+599.943703884"
Nov 03 22:32:56.634329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:56.634210    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w" podStartSLOduration=2.8484583949999998 podStartE2EDuration="5.634196822s" totalImagesPullingTime="2.785738427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:32:56.634070958 +0000 UTC m=+600.074608219" watchObservedRunningTime="2026-11-03 22:32:56.634196822 +0000 UTC m=+600.074734072"
Nov 03 22:32:56.759118 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:56.759089    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:32:56.772600 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:56.772573    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:32:59.335655 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.335618    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:32:59.688679 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.688575    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"]
Nov 03 22:32:59.693692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.693651    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:32:59.750020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.749991    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"]
Nov 03 22:32:59.837026 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.836983    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmm7f\" (UniqueName: \"kubernetes.io/projected/b455b70d-d28f-48ac-a4ef-7a5c7392d06f-kube-api-access-dmm7f\") pod \"ai-gateway-controller-5d6776c6bc-p8lvw\" (UID: \"b455b70d-d28f-48ac-a4ef-7a5c7392d06f\") " pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:32:59.837196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.837039    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b455b70d-d28f-48ac-a4ef-7a5c7392d06f-metrics-certs\") pod \"ai-gateway-controller-5d6776c6bc-p8lvw\" (UID: \"b455b70d-d28f-48ac-a4ef-7a5c7392d06f\") " pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:32:59.938465 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.938425    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmm7f\" (UniqueName: \"kubernetes.io/projected/b455b70d-d28f-48ac-a4ef-7a5c7392d06f-kube-api-access-dmm7f\") pod \"ai-gateway-controller-5d6776c6bc-p8lvw\" (UID: \"b455b70d-d28f-48ac-a4ef-7a5c7392d06f\") " pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:32:59.938635 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.938475    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b455b70d-d28f-48ac-a4ef-7a5c7392d06f-metrics-certs\") pod \"ai-gateway-controller-5d6776c6bc-p8lvw\" (UID: \"b455b70d-d28f-48ac-a4ef-7a5c7392d06f\") " pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:32:59.941118 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.941050    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b455b70d-d28f-48ac-a4ef-7a5c7392d06f-metrics-certs\") pod \"ai-gateway-controller-5d6776c6bc-p8lvw\" (UID: \"b455b70d-d28f-48ac-a4ef-7a5c7392d06f\") " pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:32:59.999462 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:32:59.999431    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmm7f\" (UniqueName: \"kubernetes.io/projected/b455b70d-d28f-48ac-a4ef-7a5c7392d06f-kube-api-access-dmm7f\") pod \"ai-gateway-controller-5d6776c6bc-p8lvw\" (UID: \"b455b70d-d28f-48ac-a4ef-7a5c7392d06f\") " pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:33:00.005805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:00.005782    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:33:00.378118 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:00.378091    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"]
Nov 03 22:33:00.380795 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:33:00.380763    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb455b70d_d28f_48ac_a4ef_7a5c7392d06f.slice/crio-7842eb0d5fb22fa98c1fd474d03895f2a1ef0a4e31c2c45295a380d52c7687d4 WatchSource:0}: Error finding container 7842eb0d5fb22fa98c1fd474d03895f2a1ef0a4e31c2c45295a380d52c7687d4: Status 404 returned error can't find the container with id 7842eb0d5fb22fa98c1fd474d03895f2a1ef0a4e31c2c45295a380d52c7687d4
Nov 03 22:33:00.400270 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:00.400240    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw" event={"ID":"b455b70d-d28f-48ac-a4ef-7a5c7392d06f","Type":"ContainerStarted","Data":"7842eb0d5fb22fa98c1fd474d03895f2a1ef0a4e31c2c45295a380d52c7687d4"}
Nov 03 22:33:01.406943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:01.406907    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw" event={"ID":"b455b70d-d28f-48ac-a4ef-7a5c7392d06f","Type":"ContainerStarted","Data":"13d0ef43fdbf5b6e789e6db2210b174e145be9db1966a793ab544e4bf0a59197"}
Nov 03 22:33:01.407388 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:01.407033    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:33:01.522927 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:01.522865    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw" podStartSLOduration=1.714333083 podStartE2EDuration="2.522849381s" totalImagesPullingTime="808.516298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:01.521381331 +0000 UTC m=+604.961918588" watchObservedRunningTime="2026-11-03 22:33:01.522849381 +0000 UTC m=+604.963386640"
Nov 03 22:33:07.388778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:07.388743    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w"
Nov 03 22:33:07.389226 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:07.388828    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-wqbz4"
Nov 03 22:33:12.413711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.413652    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:33:12.580912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.580869    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-29zcf"]
Nov 03 22:33:12.581188 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.581129    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf" podUID="bd7703d5-d970-49c1-8865-87fc8b19c2ef" containerName="manager" containerID="cri-o://dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6" gracePeriod=30
Nov 03 22:33:12.836893 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.836869    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:33:12.870164 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.870139    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd7703d5-d970-49c1-8865-87fc8b19c2ef-kube-api-access-nlb5q\" (UniqueName: \"kubernetes.io/projected/bd7703d5-d970-49c1-8865-87fc8b19c2ef-kube-api-access-nlb5q\") pod \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\" (UID: \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\") "
Nov 03 22:33:12.870322 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.870205    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd7703d5-d970-49c1-8865-87fc8b19c2ef-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7703d5-d970-49c1-8865-87fc8b19c2ef-metrics-certs\") pod \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\" (UID: \"bd7703d5-d970-49c1-8865-87fc8b19c2ef\") "
Nov 03 22:33:12.872456 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.872432    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd7703d5-d970-49c1-8865-87fc8b19c2ef-metrics-certs" pod "bd7703d5-d970-49c1-8865-87fc8b19c2ef" (UID: "bd7703d5-d970-49c1-8865-87fc8b19c2ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:33:12.872559 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.872485    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd7703d5-d970-49c1-8865-87fc8b19c2ef-kube-api-access-nlb5q" pod "bd7703d5-d970-49c1-8865-87fc8b19c2ef" (UID: "bd7703d5-d970-49c1-8865-87fc8b19c2ef"). InnerVolumeSpecName "kube-api-access-nlb5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:33:12.971602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.971570    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlb5q\" (UniqueName: \"kubernetes.io/projected/bd7703d5-d970-49c1-8865-87fc8b19c2ef-kube-api-access-nlb5q\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:33:12.971602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:12.971605    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7703d5-d970-49c1-8865-87fc8b19c2ef-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:33:13.465422 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.465388    5647 generic.go:356] "Generic (PLEG): container finished" podID="bd7703d5-d970-49c1-8865-87fc8b19c2ef" containerID="dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6" exitCode=0
Nov 03 22:33:13.465847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.465442    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf"
Nov 03 22:33:13.465847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.465471    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf" event={"ID":"bd7703d5-d970-49c1-8865-87fc8b19c2ef","Type":"ContainerDied","Data":"dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6"}
Nov 03 22:33:13.465847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.465521    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-29zcf" event={"ID":"bd7703d5-d970-49c1-8865-87fc8b19c2ef","Type":"ContainerDied","Data":"49093a288da0422321f2324c2e7c6a29ba8714893b74d93ce94d32da4370928d"}
Nov 03 22:33:13.465847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.465542    5647 scope.go:122] "RemoveContainer" containerID="dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6"
Nov 03 22:33:13.474727 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.474643    5647 scope.go:122] "RemoveContainer" containerID="dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6"
Nov 03 22:33:13.475044 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:33:13.475015    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6\": container with ID starting with dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6 not found: ID does not exist" containerID="dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6"
Nov 03 22:33:13.475153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.475059    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6"} err="failed to get container status \"dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6\": rpc error: code = NotFound desc = could not find container \"dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6\": container with ID starting with dd1569c5a163fa180a0865e4400d9272cf9e5457dccfd418f169311124de16c6 not found: ID does not exist"
Nov 03 22:33:13.628796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.628755    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-29zcf"]
Nov 03 22:33:13.633751 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:13.633726    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-29zcf"]
Nov 03 22:33:14.695384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:14.695338    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd7703d5-d970-49c1-8865-87fc8b19c2ef" path="/var/lib/kubelet/pods/bd7703d5-d970-49c1-8865-87fc8b19c2ef/volumes"
Nov 03 22:33:16.044503 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:33:16.044462    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:33:43.624461 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.624430    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"]
Nov 03 22:33:43.624918 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.624903    5647 cpu_manager.go:414] "containerMap: removing container" podUID="bd7703d5-d970-49c1-8865-87fc8b19c2ef" containerName="manager"
Nov 03 22:33:43.624918 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.624916    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd7703d5-d970-49c1-8865-87fc8b19c2ef" containerName="manager"
Nov 03 22:33:43.624987 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.624984    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd7703d5-d970-49c1-8865-87fc8b19c2ef" containerName="manager"
Nov 03 22:33:43.628189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.628167    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.680482 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.680448    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-32449d-openshift-default-dockercfg-rc6q5\""
Nov 03 22:33:43.738104 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.738055    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"]
Nov 03 22:33:43.784736 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.784690    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.784920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.784742    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.784920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.784851    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.784920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.784894    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.785065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.784985    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fb4m\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-kube-api-access-8fb4m\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.785102 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.785066    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.785143 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.785106    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31b8fab-3824-47c6-95af-922b2359f74d-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.785183 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.785156    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31b8fab-3824-47c6-95af-922b2359f74d-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.785237 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.785194    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886154 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886070    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886154 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886104    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886154 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886128    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886154 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886146    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886180    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fb4m\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-kube-api-access-8fb4m\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886205    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886230    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31b8fab-3824-47c6-95af-922b2359f74d-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886255    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31b8fab-3824-47c6-95af-922b2359f74d-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886290    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886547    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886589    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886720    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886940 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886896    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.886981 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.886933    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31b8fab-3824-47c6-95af-922b2359f74d-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.888744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.888725    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.888938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.888919    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31b8fab-3824-47c6-95af-922b2359f74d-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.949394 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.949364    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:43.952105 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:43.952082    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fb4m\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-kube-api-access-8fb4m\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:44.241206 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.241171    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:44.561617 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.561540    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"]
Nov 03 22:33:44.566295 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.566276    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.601482 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.601451    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvv55\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-kube-api-access-jvv55\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.601625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.601499    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.601625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.601530    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.601719 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.601694    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.601842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.601825    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.601908 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.601892    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e63018e-6505-4781-91e1-d261b2acafbc-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.601964 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.601947    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.602046 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.602025    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.602116 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.602100    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e63018e-6505-4781-91e1-d261b2acafbc-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.618429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.618405    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-7540da-openshift-default-dockercfg-862xt\""
Nov 03 22:33:44.625765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.625726    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"]
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.709858    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.709944    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e63018e-6505-4781-91e1-d261b2acafbc-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.709995    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.710023    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.710096    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e63018e-6505-4781-91e1-d261b2acafbc-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.710173    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvv55\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-kube-api-access-jvv55\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.710221    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.710279    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.711366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.710307    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.714067 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.712802    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e63018e-6505-4781-91e1-d261b2acafbc-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.719608 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.719554    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.720434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.720095    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.720434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.720186    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.720434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.720395    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.721090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.721036    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e63018e-6505-4781-91e1-d261b2acafbc-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.723716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.723604    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"]
Nov 03 22:33:44.724961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.724937    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.734901 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.734870    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:44.734997 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.734936    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:44.734997 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.734965    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:44.777172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.777147    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvv55\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-kube-api-access-jvv55\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.777992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.777970    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:44.881991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:44.881896    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:45.314938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.314607    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"]
Nov 03 22:33:45.317659 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:33:45.317631    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e63018e_6505_4781_91e1_d261b2acafbc.slice/crio-17796267053c3b215c90bb59cefd8dfbb18280cba9e9c9d6c82cc38b65909243 WatchSource:0}: Error finding container 17796267053c3b215c90bb59cefd8dfbb18280cba9e9c9d6c82cc38b65909243: Status 404 returned error can't find the container with id 17796267053c3b215c90bb59cefd8dfbb18280cba9e9c9d6c82cc38b65909243
Nov 03 22:33:45.329623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.329581    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:45.329777 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.329710    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:45.329777 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.329758    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:45.591112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.591039    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"]
Nov 03 22:33:45.594663 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.594643    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.637343    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.644803    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.644867    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.644933    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d8040b2-0353-449a-a8b6-01b6be112ad0-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.644954    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.645001    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2657\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-kube-api-access-s2657\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.645100    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.645178    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.647734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.645202    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.670142 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.670107    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"]
Nov 03 22:33:45.671512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.671032    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-1d31f8-openshift-default-dockercfg-wj4vg\""
Nov 03 22:33:45.677817 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.677769    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" event={"ID":"d31b8fab-3824-47c6-95af-922b2359f74d","Type":"ContainerStarted","Data":"d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36"}
Nov 03 22:33:45.678110 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.678085    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" event={"ID":"d31b8fab-3824-47c6-95af-922b2359f74d","Type":"ContainerStarted","Data":"110f40b4ce6f43f2f4804f1edaf2e380d56e925046df5ef3f1b2716a2f9336b2"}
Nov 03 22:33:45.688007 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.687976    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" event={"ID":"2e63018e-6505-4781-91e1-d261b2acafbc","Type":"ContainerStarted","Data":"f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b"}
Nov 03 22:33:45.688132 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.688030    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" event={"ID":"2e63018e-6505-4781-91e1-d261b2acafbc","Type":"ContainerStarted","Data":"17796267053c3b215c90bb59cefd8dfbb18280cba9e9c9d6c82cc38b65909243"}
Nov 03 22:33:45.745955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.745915    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2657\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-kube-api-access-s2657\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746344    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746497 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746456    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746497 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746482    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746613 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746511    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746613 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746542    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746613 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746589    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746715    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d8040b2-0353-449a-a8b6-01b6be112ad0-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746745    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.746894 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.746868    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.747611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.747581    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.747933 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.747911    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.748121 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.748104    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.748940 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.748909    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d8040b2-0353-449a-a8b6-01b6be112ad0-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.756346 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.756258    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.757740 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.757704    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.807214 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.807182    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2657\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-kube-api-access-s2657\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.807975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.807954    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.871953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.871814    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" podStartSLOduration=2.871800522 podStartE2EDuration="2.871800522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:45.871004265 +0000 UTC m=+649.311541547" watchObservedRunningTime="2026-11-03 22:33:45.871800522 +0000 UTC m=+649.312337780"
Nov 03 22:33:45.891443 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.887302    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:45.907523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.907489    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:45.994937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:45.993473    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podStartSLOduration=1.9934540090000001 podStartE2EDuration="1.993454009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:45.993146096 +0000 UTC m=+649.433683379" watchObservedRunningTime="2026-11-03 22:33:45.993454009 +0000 UTC m=+649.433991270"
Nov 03 22:33:46.050203 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:33:46.050172    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:33:46.241651 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.241619    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:46.246498 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.246475    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:46.361132 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.361099    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"]
Nov 03 22:33:46.370553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.370515    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:46.370710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.370598    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:46.370710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.370638    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:33:46.696444 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.696352    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" event={"ID":"0d8040b2-0353-449a-a8b6-01b6be112ad0","Type":"ContainerStarted","Data":"6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5"}
Nov 03 22:33:46.696444 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.696405    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" event={"ID":"0d8040b2-0353-449a-a8b6-01b6be112ad0","Type":"ContainerStarted","Data":"9dbf7c7404527f1aa5abc8cfe8822f593d3a4561fb77123d1f4d6d189a9a227c"}
Nov 03 22:33:46.696958 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.696803    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:46.697888 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.697863    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:33:46.820899 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.820846    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podStartSLOduration=1.8208290150000002 podStartE2EDuration="1.820829015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:46.819231996 +0000 UTC m=+650.259769258" watchObservedRunningTime="2026-11-03 22:33:46.820829015 +0000 UTC m=+650.261366274"
Nov 03 22:33:46.887429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.887399    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:46.908523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:46.908497    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:47.702015 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:47.701984    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:47.703013 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:47.702992    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:33:47.913935 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:47.913907    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:48.713824 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:48.713784    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:48.714785 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:48.714766    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:33:53.263940 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.263906    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"]
Nov 03 22:33:53.267820 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.267801    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:53.335025 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.330029    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx\" (UID: \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:53.335025 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.330131    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zssnm\" (UniqueName: \"kubernetes.io/projected/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-kube-api-access-zssnm\") pod \"payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx\" (UID: \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:53.394062 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.387976    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-b64e2e-dockercfg-rpwr6\""
Nov 03 22:33:53.394062 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.388153    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-b64e2e\""
Nov 03 22:33:53.394062 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.390859    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"]
Nov 03 22:33:53.431119 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.431084    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx\" (UID: \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:53.431306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.431144    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zssnm\" (UniqueName: \"kubernetes.io/projected/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-kube-api-access-zssnm\") pod \"payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx\" (UID: \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:53.431778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.431758    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx\" (UID: \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:53.505281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.505244    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"]
Nov 03 22:33:53.511564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.511538    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:53.518376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.518314    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zssnm\" (UniqueName: \"kubernetes.io/projected/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-kube-api-access-zssnm\") pod \"payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx\" (UID: \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:53.532400 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.532337    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxqbg\" (UniqueName: \"kubernetes.io/projected/8576b4d8-ff39-4559-aa81-8570063bfe4a-kube-api-access-hxqbg\") pod \"payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds\" (UID: \"8576b4d8-ff39-4559-aa81-8570063bfe4a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:53.532546 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.532523    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8576b4d8-ff39-4559-aa81-8570063bfe4a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds\" (UID: \"8576b4d8-ff39-4559-aa81-8570063bfe4a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:53.578777 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.578729    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"]
Nov 03 22:33:53.581308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.581281    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:53.633535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.633488    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8576b4d8-ff39-4559-aa81-8570063bfe4a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds\" (UID: \"8576b4d8-ff39-4559-aa81-8570063bfe4a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:53.633645 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.633623    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxqbg\" (UniqueName: \"kubernetes.io/projected/8576b4d8-ff39-4559-aa81-8570063bfe4a-kube-api-access-hxqbg\") pod \"payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds\" (UID: \"8576b4d8-ff39-4559-aa81-8570063bfe4a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:53.634541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.634480    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8576b4d8-ff39-4559-aa81-8570063bfe4a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds\" (UID: \"8576b4d8-ff39-4559-aa81-8570063bfe4a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:53.698711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.698661    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxqbg\" (UniqueName: \"kubernetes.io/projected/8576b4d8-ff39-4559-aa81-8570063bfe4a-kube-api-access-hxqbg\") pod \"payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds\" (UID: \"8576b4d8-ff39-4559-aa81-8570063bfe4a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:53.833619 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.833532    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:53.952977 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:53.952948    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"]
Nov 03 22:33:53.954870 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:33:53.954840    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c9ed8a2_ef4f_41f1_a896_bf1cd7bf1198.slice/crio-d88ceed0f6655110127383bb0a1708cfbd0f173c548d06d435ef46d976224ae0 WatchSource:0}: Error finding container d88ceed0f6655110127383bb0a1708cfbd0f173c548d06d435ef46d976224ae0: Status 404 returned error can't find the container with id d88ceed0f6655110127383bb0a1708cfbd0f173c548d06d435ef46d976224ae0
Nov 03 22:33:54.205252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.205225    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"]
Nov 03 22:33:54.207461 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:33:54.207432    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8576b4d8_ff39_4559_aa81_8570063bfe4a.slice/crio-fd45ea3599a35aa6ab078331ac18419fea321c213cf8bebfa6584f829e3439dc WatchSource:0}: Error finding container fd45ea3599a35aa6ab078331ac18419fea321c213cf8bebfa6584f829e3439dc: Status 404 returned error can't find the container with id fd45ea3599a35aa6ab078331ac18419fea321c213cf8bebfa6584f829e3439dc
Nov 03 22:33:54.744371 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.744332    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds" event={"ID":"8576b4d8-ff39-4559-aa81-8570063bfe4a","Type":"ContainerStarted","Data":"4bb3910ea2294d45bb65e17eae2e4f64af2edcc3b58eaf22ce707585da4105c3"}
Nov 03 22:33:54.744876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.744391    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds" event={"ID":"8576b4d8-ff39-4559-aa81-8570063bfe4a","Type":"ContainerStarted","Data":"fd45ea3599a35aa6ab078331ac18419fea321c213cf8bebfa6584f829e3439dc"}
Nov 03 22:33:54.744876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.744506    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:33:54.746002 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.745979    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx" event={"ID":"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198","Type":"ContainerStarted","Data":"474a49cf561c6d0bef72bd6db96860c2b123a750bf2535ab355887c361ddcdff"}
Nov 03 22:33:54.746128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.746017    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx" event={"ID":"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198","Type":"ContainerStarted","Data":"d88ceed0f6655110127383bb0a1708cfbd0f173c548d06d435ef46d976224ae0"}
Nov 03 22:33:54.746128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.746038    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:33:54.861233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.861176    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds" podStartSLOduration=1.861162276 podStartE2EDuration="1.861162276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:54.860542842 +0000 UTC m=+658.301080099" watchObservedRunningTime="2026-11-03 22:33:54.861162276 +0000 UTC m=+658.301699535"
Nov 03 22:33:54.976578 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:54.976518    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx" podStartSLOduration=1.976503892 podStartE2EDuration="1.976503892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:54.974950593 +0000 UTC m=+658.415487854" watchObservedRunningTime="2026-11-03 22:33:54.976503892 +0000 UTC m=+658.417041200"
Nov 03 22:33:58.183701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.183640    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"]
Nov 03 22:33:58.188807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.188785    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:58.244252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.244220    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"]
Nov 03 22:33:58.291916 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.291886    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-32449d\""
Nov 03 22:33:58.292247 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.292228    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-32449d-dockercfg-znv2m\""
Nov 03 22:33:58.292549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.292510    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfd62750-e701-4dd2-8693-752677fafdd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf\" (UID: \"dfd62750-e701-4dd2-8693-752677fafdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:58.292648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.292627    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njxhd\" (UniqueName: \"kubernetes.io/projected/dfd62750-e701-4dd2-8693-752677fafdd4-kube-api-access-njxhd\") pod \"payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf\" (UID: \"dfd62750-e701-4dd2-8693-752677fafdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:58.340765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.340729    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"]
Nov 03 22:33:58.344392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.344377    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:58.393886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.393853    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njxhd\" (UniqueName: \"kubernetes.io/projected/dfd62750-e701-4dd2-8693-752677fafdd4-kube-api-access-njxhd\") pod \"payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf\" (UID: \"dfd62750-e701-4dd2-8693-752677fafdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:58.394044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.393898    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxq6g\" (UniqueName: \"kubernetes.io/projected/d0fb062c-7b7b-42c1-a99d-978d35c02be9-kube-api-access-fxq6g\") pod \"payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h\" (UID: \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:58.394044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.394016    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0fb062c-7b7b-42c1-a99d-978d35c02be9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h\" (UID: \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:58.394257 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.394234    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfd62750-e701-4dd2-8693-752677fafdd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf\" (UID: \"dfd62750-e701-4dd2-8693-752677fafdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:58.394905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.394888    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfd62750-e701-4dd2-8693-752677fafdd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf\" (UID: \"dfd62750-e701-4dd2-8693-752677fafdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:58.401886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.401864    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"]
Nov 03 22:33:58.454335 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.454316    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njxhd\" (UniqueName: \"kubernetes.io/projected/dfd62750-e701-4dd2-8693-752677fafdd4-kube-api-access-njxhd\") pod \"payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf\" (UID: \"dfd62750-e701-4dd2-8693-752677fafdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:58.495604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.495545    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxq6g\" (UniqueName: \"kubernetes.io/projected/d0fb062c-7b7b-42c1-a99d-978d35c02be9-kube-api-access-fxq6g\") pod \"payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h\" (UID: \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:58.495785 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.495657    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0fb062c-7b7b-42c1-a99d-978d35c02be9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h\" (UID: \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:58.496569 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.496526    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0fb062c-7b7b-42c1-a99d-978d35c02be9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h\" (UID: \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:58.502000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.501976    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:58.557794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.557266    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxq6g\" (UniqueName: \"kubernetes.io/projected/d0fb062c-7b7b-42c1-a99d-978d35c02be9-kube-api-access-fxq6g\") pod \"payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h\" (UID: \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:58.657115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.657084    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:58.873059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:58.872994    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"]
Nov 03 22:33:58.875612 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:33:58.875581    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd62750_e701_4dd2_8693_752677fafdd4.slice/crio-b027f5be63ff32ad017a7b9538e2945a65265b963042995ae54ee01a2a4a2cb6 WatchSource:0}: Error finding container b027f5be63ff32ad017a7b9538e2945a65265b963042995ae54ee01a2a4a2cb6: Status 404 returned error can't find the container with id b027f5be63ff32ad017a7b9538e2945a65265b963042995ae54ee01a2a4a2cb6
Nov 03 22:33:59.029449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:59.029393    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"]
Nov 03 22:33:59.031815 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:33:59.031788    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0fb062c_7b7b_42c1_a99d_978d35c02be9.slice/crio-e2da178550f97f41a50d6583dbd15046b845e4664d17d8c7469d645ad145e235 WatchSource:0}: Error finding container e2da178550f97f41a50d6583dbd15046b845e4664d17d8c7469d645ad145e235: Status 404 returned error can't find the container with id e2da178550f97f41a50d6583dbd15046b845e4664d17d8c7469d645ad145e235
Nov 03 22:33:59.772738 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:59.772694    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h" event={"ID":"d0fb062c-7b7b-42c1-a99d-978d35c02be9","Type":"ContainerStarted","Data":"89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133"}
Nov 03 22:33:59.773221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:59.772758    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h" event={"ID":"d0fb062c-7b7b-42c1-a99d-978d35c02be9","Type":"ContainerStarted","Data":"e2da178550f97f41a50d6583dbd15046b845e4664d17d8c7469d645ad145e235"}
Nov 03 22:33:59.773221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:59.772815    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:33:59.774304 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:59.774283    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf" event={"ID":"dfd62750-e701-4dd2-8693-752677fafdd4","Type":"ContainerStarted","Data":"d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b"}
Nov 03 22:33:59.774431 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:59.774328    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf" event={"ID":"dfd62750-e701-4dd2-8693-752677fafdd4","Type":"ContainerStarted","Data":"b027f5be63ff32ad017a7b9538e2945a65265b963042995ae54ee01a2a4a2cb6"}
Nov 03 22:33:59.774431 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:59.774354    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:33:59.903068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:33:59.903015    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h" podStartSLOduration=1.90300072 podStartE2EDuration="1.90300072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:33:59.901474224 +0000 UTC m=+663.342011484" watchObservedRunningTime="2026-11-03 22:33:59.90300072 +0000 UTC m=+663.343538011"
Nov 03 22:34:00.017520 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:00.017462    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf" podStartSLOduration=2.017447446 podStartE2EDuration="2.017447446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:33:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:00.015573049 +0000 UTC m=+663.456110310" watchObservedRunningTime="2026-11-03 22:34:00.017447446 +0000 UTC m=+663.457984731"
Nov 03 22:34:03.187804 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:03.187769    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:34:05.391966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.371510    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"]
Nov 03 22:34:05.399469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.399442    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.431565 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.431532    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"]
Nov 03 22:34:05.494968 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.494936    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-7540da-serving-cert\""
Nov 03 22:34:05.513697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.513215    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qhwz\" (UniqueName: \"kubernetes.io/projected/09660488-c079-4e23-92d9-59c970651afe-kube-api-access-9qhwz\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.513697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.513312    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-metrics-certs\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.513697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.513353    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.513697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.513391    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09660488-c079-4e23-92d9-59c970651afe-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.614429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.614399    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-metrics-certs\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.614645 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.614438    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.614645 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.614466    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09660488-c079-4e23-92d9-59c970651afe-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.614645 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.614554    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qhwz\" (UniqueName: \"kubernetes.io/projected/09660488-c079-4e23-92d9-59c970651afe-kube-api-access-9qhwz\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.614645 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:34:05.614581    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-serving-cert: secret "maas-api-e2e-worker-w6-7540da-serving-cert" not found
Nov 03 22:34:05.614920 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:34:05.614697    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls podName:09660488-c079-4e23-92d9-59c970651afe nodeName:}" failed. No retries permitted until 2026-11-03 22:34:06.114657102 +0000 UTC m=+669.555194338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls") pod "maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f" (UID: "09660488-c079-4e23-92d9-59c970651afe") : secret "maas-api-e2e-worker-w6-7540da-serving-cert" not found
Nov 03 22:34:05.615234 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.615216    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09660488-c079-4e23-92d9-59c970651afe-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.617030 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.617007    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-metrics-certs\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.699492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.699404    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qhwz\" (UniqueName: \"kubernetes.io/projected/09660488-c079-4e23-92d9-59c970651afe-kube-api-access-9qhwz\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:05.754658 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.754631    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 22:34:05.754812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:05.754726    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 22:34:06.129988 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:06.129957    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:06.132453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:06.132429    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls\") pod \"maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:06.314052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:06.314019    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:06.740543 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:06.740520    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"]
Nov 03 22:34:06.742695 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:34:06.742652    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09660488_c079_4e23_92d9_59c970651afe.slice/crio-4040a871c68625ad299a667eab084292b2ed196ba59d82e2128d464eaae571dc WatchSource:0}: Error finding container 4040a871c68625ad299a667eab084292b2ed196ba59d82e2128d464eaae571dc: Status 404 returned error can't find the container with id 4040a871c68625ad299a667eab084292b2ed196ba59d82e2128d464eaae571dc
Nov 03 22:34:06.744041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:06.744024    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:34:06.842523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:06.842492    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f" event={"ID":"09660488-c079-4e23-92d9-59c970651afe","Type":"ContainerStarted","Data":"4040a871c68625ad299a667eab084292b2ed196ba59d82e2128d464eaae571dc"}
Nov 03 22:34:07.849354 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:07.849321    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f" event={"ID":"09660488-c079-4e23-92d9-59c970651afe","Type":"ContainerStarted","Data":"596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6"}
Nov 03 22:34:07.849781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:07.849409    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:07.999410 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:07.999347    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f" podStartSLOduration=2.128632779 podStartE2EDuration="2.999325738s" totalImagesPullingTime="870.692959ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:07.992176906 +0000 UTC m=+671.432714166" watchObservedRunningTime="2026-11-03 22:34:07.999325738 +0000 UTC m=+671.439863002"
Nov 03 22:34:10.788838 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:10.788809    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 22:34:10.789294 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:10.788875    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 22:34:12.296526 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.296490    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:34:12.881193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.881146    5647 generic.go:356] "Generic (PLEG): container finished" podID="d31b8fab-3824-47c6-95af-922b2359f74d" containerID="d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36" exitCode=0
Nov 03 22:34:12.881487 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.881227    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" event={"ID":"d31b8fab-3824-47c6-95af-922b2359f74d","Type":"ContainerDied","Data":"d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36"}
Nov 03 22:34:12.881732 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.881713    5647 scope.go:122] "RemoveContainer" containerID="d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36"
Nov 03 22:34:12.883187 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.883153    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:12.883282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.883252    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:12.883324 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.883299    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:12.986420 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.986385    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"]
Nov 03 22:34:12.989222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:12.989197    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:13.036697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.036597    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"]
Nov 03 22:34:13.052077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.052048    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-cbd26c-dockercfg-vw9rk\""
Nov 03 22:34:13.054315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.053908    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-cbd26c\""
Nov 03 22:34:13.122070 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.122013    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"]
Nov 03 22:34:13.127117 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.127085    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:13.145915 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.145882    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf6t6\" (UniqueName: \"kubernetes.io/projected/b42582d3-f555-4fc5-8e82-732be9385712-kube-api-access-lf6t6\") pod \"payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg\" (UID: \"b42582d3-f555-4fc5-8e82-732be9385712\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:13.146068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.145939    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b42582d3-f555-4fc5-8e82-732be9385712-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg\" (UID: \"b42582d3-f555-4fc5-8e82-732be9385712\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:13.183898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.183865    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"]
Nov 03 22:34:13.247574 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.247530    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf6t6\" (UniqueName: \"kubernetes.io/projected/b42582d3-f555-4fc5-8e82-732be9385712-kube-api-access-lf6t6\") pod \"payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg\" (UID: \"b42582d3-f555-4fc5-8e82-732be9385712\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:13.247799 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.247609    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bglz\" (UniqueName: \"kubernetes.io/projected/84e7a354-3e83-4cec-93e1-50f034204374-kube-api-access-5bglz\") pod \"payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8\" (UID: \"84e7a354-3e83-4cec-93e1-50f034204374\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:13.247799 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.247682    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b42582d3-f555-4fc5-8e82-732be9385712-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg\" (UID: \"b42582d3-f555-4fc5-8e82-732be9385712\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:13.247799 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.247783    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84e7a354-3e83-4cec-93e1-50f034204374-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8\" (UID: \"84e7a354-3e83-4cec-93e1-50f034204374\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:13.248624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.248594    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b42582d3-f555-4fc5-8e82-732be9385712-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg\" (UID: \"b42582d3-f555-4fc5-8e82-732be9385712\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:13.311850 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.311818    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf6t6\" (UniqueName: \"kubernetes.io/projected/b42582d3-f555-4fc5-8e82-732be9385712-kube-api-access-lf6t6\") pod \"payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg\" (UID: \"b42582d3-f555-4fc5-8e82-732be9385712\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:13.349416 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.349381    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bglz\" (UniqueName: \"kubernetes.io/projected/84e7a354-3e83-4cec-93e1-50f034204374-kube-api-access-5bglz\") pod \"payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8\" (UID: \"84e7a354-3e83-4cec-93e1-50f034204374\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:13.349622 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.349466    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84e7a354-3e83-4cec-93e1-50f034204374-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8\" (UID: \"84e7a354-3e83-4cec-93e1-50f034204374\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:13.350215 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.350193    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84e7a354-3e83-4cec-93e1-50f034204374-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8\" (UID: \"84e7a354-3e83-4cec-93e1-50f034204374\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:13.421745 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.420984    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bglz\" (UniqueName: \"kubernetes.io/projected/84e7a354-3e83-4cec-93e1-50f034204374-kube-api-access-5bglz\") pod \"payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8\" (UID: \"84e7a354-3e83-4cec-93e1-50f034204374\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:13.444381 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.443655    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:13.639760 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.611969    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:13.863086 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.863030    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 22:34:13.869471 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.869407    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"]
Nov 03 22:34:13.933408 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:34:13.933372    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e7a354_3e83_4cec_93e1_50f034204374.slice/crio-4e03185585cbad2ceb89de39185ce4f329a2985be3747ca5f65a790cc2e0b477 WatchSource:0}: Error finding container 4e03185585cbad2ceb89de39185ce4f329a2985be3747ca5f65a790cc2e0b477: Status 404 returned error can't find the container with id 4e03185585cbad2ceb89de39185ce4f329a2985be3747ca5f65a790cc2e0b477
Nov 03 22:34:13.938107 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:13.938077    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" event={"ID":"d31b8fab-3824-47c6-95af-922b2359f74d","Type":"ContainerStarted","Data":"2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5"}
Nov 03 22:34:14.138479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.138448    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"]
Nov 03 22:34:14.144262 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:34:14.144225    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42582d3_f555_4fc5_8e82_732be9385712.slice/crio-d395433bb7d98c611c4de104517e9ddfbeb5c20eae763393aa3087eb01aa62bb WatchSource:0}: Error finding container d395433bb7d98c611c4de104517e9ddfbeb5c20eae763393aa3087eb01aa62bb: Status 404 returned error can't find the container with id d395433bb7d98c611c4de104517e9ddfbeb5c20eae763393aa3087eb01aa62bb
Nov 03 22:34:14.241482 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.241462    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:34:14.242323 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.242301    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:34:14.242434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.242350    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:34:14.945597 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.945557    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8" event={"ID":"84e7a354-3e83-4cec-93e1-50f034204374","Type":"ContainerStarted","Data":"a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f"}
Nov 03 22:34:14.946026 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.945659    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8" event={"ID":"84e7a354-3e83-4cec-93e1-50f034204374","Type":"ContainerStarted","Data":"4e03185585cbad2ceb89de39185ce4f329a2985be3747ca5f65a790cc2e0b477"}
Nov 03 22:34:14.946026 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.945824    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:14.947586 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.947566    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg" event={"ID":"b42582d3-f555-4fc5-8e82-732be9385712","Type":"ContainerStarted","Data":"5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e"}
Nov 03 22:34:14.947658 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.947602    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg" event={"ID":"b42582d3-f555-4fc5-8e82-732be9385712","Type":"ContainerStarted","Data":"d395433bb7d98c611c4de104517e9ddfbeb5c20eae763393aa3087eb01aa62bb"}
Nov 03 22:34:14.947794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:14.947784    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:15.069306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:15.069251    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8" podStartSLOduration=2.069236898 podStartE2EDuration="2.069236898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:15.067740905 +0000 UTC m=+678.508278164" watchObservedRunningTime="2026-11-03 22:34:15.069236898 +0000 UTC m=+678.509774230"
Nov 03 22:34:15.213695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:15.209529    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg" podStartSLOduration=3.209508236 podStartE2EDuration="3.209508236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:15.207068324 +0000 UTC m=+678.647605584" watchObservedRunningTime="2026-11-03 22:34:15.209508236 +0000 UTC m=+678.650045495"
Nov 03 22:34:15.241516 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:15.241486    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:34:15.241814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:15.241790    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:34:16.086703 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:34:16.086601    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:34:16.242611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:16.242563    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:34:16.242802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:16.242627    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:34:16.723341 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:16.723115    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:34:17.241783 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:17.241747    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:34:17.242179 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:17.241803    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:34:18.247831 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:18.247783    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Nov 03 22:34:18.248359 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:18.247896    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Nov 03 22:34:19.246334 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:19.246304    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:34:19.247353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:19.247334    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 22:34:20.789297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:20.789264    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"]
Nov 03 22:34:20.792597 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:20.792578    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:20.844349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:20.844316    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-b64e2e-serving-cert\""
Nov 03 22:34:20.897245 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:20.897208    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"]
Nov 03 22:34:20.958355 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:20.958327    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:20.958515 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:20.958438    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9x44\" (UniqueName: \"kubernetes.io/projected/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-kube-api-access-r9x44\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:20.958564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:20.958531    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:20.958608 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:20.958579    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-metrics-certs\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.059726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.059582    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.059726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.059698    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9x44\" (UniqueName: \"kubernetes.io/projected/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-kube-api-access-r9x44\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.060027 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.059911    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.060027 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.059946    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-metrics-certs\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.063592 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:34:21.061201    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-serving-cert: secret "maas-api-e2e-worker-w0-b64e2e-serving-cert" not found
Nov 03 22:34:21.063592 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:34:21.061271    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls podName:4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863 nodeName:}" failed. No retries permitted until 2026-11-03 22:34:21.561252168 +0000 UTC m=+685.001789406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls") pod "maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b" (UID: "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863") : secret "maas-api-e2e-worker-w0-b64e2e-serving-cert" not found
Nov 03 22:34:21.063592 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.062238    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.075520 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.075484    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-metrics-certs\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.124707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.124407    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9x44\" (UniqueName: \"kubernetes.io/projected/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-kube-api-access-r9x44\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.205395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.205356    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:34:21.565655 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.565605    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.568128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.568101    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:21.705485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:21.705444    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:22.080429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:22.080392    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"]
Nov 03 22:34:22.083074 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:34:22.083040    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e81333e_ec9f_4f3a_8ff2_b8dc6d2bd863.slice/crio-b70e775b8c0dcb5f0bf37bdde3c3124692a2d94d6489d49a8a25149d97c7e391 WatchSource:0}: Error finding container b70e775b8c0dcb5f0bf37bdde3c3124692a2d94d6489d49a8a25149d97c7e391: Status 404 returned error can't find the container with id b70e775b8c0dcb5f0bf37bdde3c3124692a2d94d6489d49a8a25149d97c7e391
Nov 03 22:34:23.011415 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.011364    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b" event={"ID":"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863","Type":"ContainerStarted","Data":"5c42ff4696521824f792d99bffb1ba0ba17d1c3fbc081a13918b624e223ce555"}
Nov 03 22:34:23.011646 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.011470    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b" event={"ID":"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863","Type":"ContainerStarted","Data":"b70e775b8c0dcb5f0bf37bdde3c3124692a2d94d6489d49a8a25149d97c7e391"}
Nov 03 22:34:23.011646 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.011500    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:23.013160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.013138    5647 generic.go:356] "Generic (PLEG): container finished" podID="2e63018e-6505-4781-91e1-d261b2acafbc" containerID="f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b" exitCode=0
Nov 03 22:34:23.013269 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.013181    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" event={"ID":"2e63018e-6505-4781-91e1-d261b2acafbc","Type":"ContainerDied","Data":"f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b"}
Nov 03 22:34:23.013454 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.013441    5647 scope.go:122] "RemoveContainer" containerID="f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b"
Nov 03 22:34:23.014646 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.014620    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:23.014732 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.014698    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:23.014768 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.014730    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:23.156209 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:23.156082    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b" podStartSLOduration=2.371336658 podStartE2EDuration="3.156066993s" totalImagesPullingTime="784.730335ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:23.154267057 +0000 UTC m=+686.594804316" watchObservedRunningTime="2026-11-03 22:34:23.156066993 +0000 UTC m=+686.596604267"
Nov 03 22:34:24.041795 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:24.041758    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" event={"ID":"2e63018e-6505-4781-91e1-d261b2acafbc","Type":"ContainerStarted","Data":"9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73"}
Nov 03 22:34:24.890694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:24.886083    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:34:24.890694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:24.886215    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:34:24.890694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:24.886268    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:34:25.889693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:25.889021    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:34:25.889693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:25.889102    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:34:25.956227 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:25.956195    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:34:25.956616 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:25.956403    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:34:26.890695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:26.888119    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:34:26.890695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:26.888190    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:34:27.888113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:27.888072    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:34:27.888772 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:27.888739    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:34:28.893692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:28.893074    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Nov 03 22:34:28.893692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:28.893140    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Nov 03 22:34:29.048738 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:29.048699    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 22:34:29.886749 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:29.886711    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:34:29.887089 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:29.887070    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:34:29.887709 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:29.887690    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 22:34:30.432193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.432164    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"]
Nov 03 22:34:30.435656 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.435636    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.492936 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.492894    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.493129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.492955    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.493129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.493066    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.493253 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.493126    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.493253 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.493220    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5jp7\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-kube-api-access-d5jp7\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.493354 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.493273    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.493354 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.493310    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.493426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.493382    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.493426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.493405    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.525287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.525255    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"]
Nov 03 22:34:30.575812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.575783    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-164c36-openshift-default-dockercfg-tlv7g\""
Nov 03 22:34:30.594279 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594255    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594438 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594290    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594438 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594310    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594437    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594495    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594543    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594740 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594581    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594740 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594627    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5jp7\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-kube-api-access-d5jp7\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594740 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594709    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594867 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594756    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.594867 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.594844    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.595059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.595041    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.595140 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.595117    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.595258 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.595243    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.596781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.596760    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.597497 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.597415    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.655174 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.655139    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.655346 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.655256    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5jp7\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-kube-api-access-d5jp7\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:30.747614 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:30.747582    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:31.175557 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:31.175532    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"]
Nov 03 22:34:31.177618 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:34:31.177597    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf4759b_d708_48e1_9ac2_7ce04f607b9b.slice/crio-7b66c1591ca38878adb0dd86597c959a52382d7910d63de8d7c009d2fb6aabbb WatchSource:0}: Error finding container 7b66c1591ca38878adb0dd86597c959a52382d7910d63de8d7c009d2fb6aabbb: Status 404 returned error can't find the container with id 7b66c1591ca38878adb0dd86597c959a52382d7910d63de8d7c009d2fb6aabbb
Nov 03 22:34:31.180031 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:31.179999    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:31.180117 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:31.180064    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:31.180117 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:31.180096    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:32.101254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:32.101218    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" event={"ID":"fbf4759b-d708-48e1-9ac2-7ce04f607b9b","Type":"ContainerStarted","Data":"77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483"}
Nov 03 22:34:32.101806 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:32.101274    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" event={"ID":"fbf4759b-d708-48e1-9ac2-7ce04f607b9b","Type":"ContainerStarted","Data":"7b66c1591ca38878adb0dd86597c959a52382d7910d63de8d7c009d2fb6aabbb"}
Nov 03 22:34:32.290690 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:32.290074    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" podStartSLOduration=2.290056324 podStartE2EDuration="2.290056324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:34:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:34:32.288004898 +0000 UTC m=+695.728542157" watchObservedRunningTime="2026-11-03 22:34:32.290056324 +0000 UTC m=+695.730593584"
Nov 03 22:34:32.748285 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:32.748252    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:32.749859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:32.749833    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 03 22:34:32.749982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:32.749881    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 03 22:34:33.748244 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:33.748212    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 03 22:34:33.748691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:33.748262    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 03 22:34:34.747916 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:34.747881    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 03 22:34:34.748095 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:34.747929    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 03 22:34:35.748461 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:35.748425    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 03 22:34:35.748889 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:35.748482    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 03 22:34:36.748127 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:36.748087    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 03 22:34:36.748306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:36.748140    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 03 22:34:37.752791 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:37.752761    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:37.753212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:37.753073    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:37.753730 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:37.753711    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:34:38.596636 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:38.596600    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:34:46.046318 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:34:46.046272    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:34:48.714820 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:48.714782    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:34:48.715259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:48.714847    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:34:49.197411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:49.197326    5647 generic.go:356] "Generic (PLEG): container finished" podID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerID="6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5" exitCode=0
Nov 03 22:34:49.197411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:49.197393    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" event={"ID":"0d8040b2-0353-449a-a8b6-01b6be112ad0","Type":"ContainerDied","Data":"6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5"}
Nov 03 22:34:49.197863 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:49.197844    5647 scope.go:122] "RemoveContainer" containerID="6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5"
Nov 03 22:34:49.199202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:49.199174    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:49.199265 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:49.199245    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:49.199299 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:49.199282    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:34:50.212215 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:50.212181    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" event={"ID":"0d8040b2-0353-449a-a8b6-01b6be112ad0","Type":"ContainerStarted","Data":"6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2"}
Nov 03 22:34:50.907826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:50.907793    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:34:50.908131 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:50.908107    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:34:50.908195 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:50.908173    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:34:51.908751 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:51.908698    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:34:51.909159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:51.908765    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:34:52.908152 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:52.908121    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:34:52.908349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:52.908171    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:34:53.908232 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:53.908196    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:34:53.908740 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:53.908250    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:34:54.908641 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:54.908603    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 03 22:34:54.909072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:54.908680    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 03 22:34:55.913164 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:55.913130    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:34:55.913624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:55.913436    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:34:55.914262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:55.914246    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 22:34:58.881875 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:34:58.881835    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:35:00.674028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.673980    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"]
Nov 03 22:35:00.677438 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.677421    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.771555 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.771520    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"]
Nov 03 22:35:00.776166 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776135    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.776166 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776167    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.776401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776199    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6104501a-1b6d-4e27-8838-3cc5e45dffee-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.776401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776340    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.776517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776417    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.776517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776498    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.776623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776533    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gdj8\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-kube-api-access-5gdj8\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.776623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776572    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.776780 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.776762    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.878201 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.878159    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.878465 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.878445    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.878624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.878583    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.878624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.878617    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gdj8\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-kube-api-access-5gdj8\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.878908 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.878661    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.878908 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.878847    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.878908 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.878902    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.879080 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.878960    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.879080 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.879044    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6104501a-1b6d-4e27-8838-3cc5e45dffee-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.883305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.882278    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6104501a-1b6d-4e27-8838-3cc5e45dffee-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.883305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.883045    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.883305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.883318    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.883618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.883561    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.884207 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.884181    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.905350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.905320    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.925873 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.925767    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.944338 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.944309    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gdj8\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-kube-api-access-5gdj8\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.944525 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.944418    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:00.990066 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:00.990036    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:01.420059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:01.420031    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"]
Nov 03 22:35:01.422345 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:35:01.422316    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6104501a_1b6d_4e27_8838_3cc5e45dffee.slice/crio-b56f6240f129258c457c9e6d014b1ce92f9739ccd710c2125b57b1be9fae6155 WatchSource:0}: Error finding container b56f6240f129258c457c9e6d014b1ce92f9739ccd710c2125b57b1be9fae6155: Status 404 returned error can't find the container with id b56f6240f129258c457c9e6d014b1ce92f9739ccd710c2125b57b1be9fae6155
Nov 03 22:35:01.424778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:01.424742    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:35:01.424854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:01.424820    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:35:01.424898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:01.424854    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:35:02.299835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:02.299793    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" event={"ID":"6104501a-1b6d-4e27-8838-3cc5e45dffee","Type":"ContainerStarted","Data":"2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6"}
Nov 03 22:35:02.300339 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:02.299848    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" event={"ID":"6104501a-1b6d-4e27-8838-3cc5e45dffee","Type":"ContainerStarted","Data":"b56f6240f129258c457c9e6d014b1ce92f9739ccd710c2125b57b1be9fae6155"}
Nov 03 22:35:02.468048 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:02.467979    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podStartSLOduration=2.467961215 podStartE2EDuration="2.467961215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:35:02.467579212 +0000 UTC m=+725.908116496" watchObservedRunningTime="2026-11-03 22:35:02.467961215 +0000 UTC m=+725.908498475"
Nov 03 22:35:02.995058 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:02.995023    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:02.997707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:02.997326    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 03 22:35:02.997707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:02.997386    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 03 22:35:03.995804 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:03.995758    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 03 22:35:03.996317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:03.995817    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 03 22:35:04.995880 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:04.995827    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 03 22:35:04.996315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:04.995895    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 03 22:35:05.993547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:05.993112    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 03 22:35:05.993547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:05.993174    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 03 22:35:06.993211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:06.993118    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Nov 03 22:35:06.994945 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:06.994911    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Nov 03 22:35:07.995356 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:07.995328    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:07.995801 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:07.995689    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:07.996283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:07.996266    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:35:13.211481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:13.211438    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:35:16.042835 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:35:16.042800    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:35:17.790517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.790478    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"]
Nov 03 22:35:17.794549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.794530    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:17.895942 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.895904    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-7540da\"/\"kube-root-ca.crt\""
Nov 03 22:35:17.895942 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.895907    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-7540da\"/\"openshift-service-ca.crt\""
Nov 03 22:35:17.896167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.895979    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-7540da\"/\"default-dockercfg-7hxtx\""
Nov 03 22:35:17.896167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.895981    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-7540da\"/\"simulator-w6-7540da-kserve-self-signed-certs\""
Nov 03 22:35:17.898198 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.898174    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"]
Nov 03 22:35:17.902945 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.902923    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-dshm\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:17.903082 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.902969    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28f92588-8375-4ef4-ae91-3ff783e71def-tls-certs\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:17.903082 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.903067    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-home\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:17.903179 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.903120    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfvbn\" (UniqueName: \"kubernetes.io/projected/28f92588-8375-4ef4-ae91-3ff783e71def-kube-api-access-sfvbn\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:17.903236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.903222    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-tmp-dir\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:17.903277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:17.903251    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-model-cache\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.004644 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.004605    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28f92588-8375-4ef4-ae91-3ff783e71def-tls-certs\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.004644 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.004647    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-home\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.004917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.004703    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfvbn\" (UniqueName: \"kubernetes.io/projected/28f92588-8375-4ef4-ae91-3ff783e71def-kube-api-access-sfvbn\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.004917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.004750    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-tmp-dir\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.004917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.004873    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-model-cache\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.005088 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.005023    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-dshm\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.005210 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.005187    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-home\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.005274 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.005257    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-tmp-dir\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.005329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.005308    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-model-cache\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.007289 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.007265    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-dshm\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.007388 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.007355    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28f92588-8375-4ef4-ae91-3ff783e71def-tls-certs\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.070115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.070039    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfvbn\" (UniqueName: \"kubernetes.io/projected/28f92588-8375-4ef4-ae91-3ff783e71def-kube-api-access-sfvbn\") pod \"simulator-w6-7540da-kserve-5567d5c5d8-8ncnw\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") " pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.107157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.107122    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:18.485085 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:18.485059    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"]
Nov 03 22:35:18.487465 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:35:18.487438    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28f92588_8375_4ef4_ae91_3ff783e71def.slice/crio-eab7676579c60a3b08154b8005b5e8d2dab80b78ea57e73357dc12e4d52d191b WatchSource:0}: Error finding container eab7676579c60a3b08154b8005b5e8d2dab80b78ea57e73357dc12e4d52d191b: Status 404 returned error can't find the container with id eab7676579c60a3b08154b8005b5e8d2dab80b78ea57e73357dc12e4d52d191b
Nov 03 22:35:19.412591 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:19.412555    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" event={"ID":"28f92588-8375-4ef4-ae91-3ff783e71def","Type":"ContainerStarted","Data":"f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57"}
Nov 03 22:35:19.412591 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:19.412604    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" event={"ID":"28f92588-8375-4ef4-ae91-3ff783e71def","Type":"ContainerStarted","Data":"eab7676579c60a3b08154b8005b5e8d2dab80b78ea57e73357dc12e4d52d191b"}
Nov 03 22:35:19.530224 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:19.530158    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podStartSLOduration=2.530145956 podStartE2EDuration="2.530145956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:35:19.527724549 +0000 UTC m=+742.968261808" watchObservedRunningTime="2026-11-03 22:35:19.530145956 +0000 UTC m=+742.970683214"
Nov 03 22:35:20.107580 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:20.107530    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:28.108051 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:28.108013    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:28.121026 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:28.121001    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:28.508729 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:28.508701    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:35:33.163330 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:33.163294    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:35:35.306424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.306388    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"]
Nov 03 22:35:35.311869 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.311844    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.366056 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.366016    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-model-cache\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.366250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.366067    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86774a7f-0cd5-43d6-9162-c9bf260453ed-tls-certs\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.366250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.366223    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-home\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.366360 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.366274    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-dshm\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.366397 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.366384    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-tmp-dir\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.366462 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.366443    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdwct\" (UniqueName: \"kubernetes.io/projected/86774a7f-0cd5-43d6-9162-c9bf260453ed-kube-api-access-mdwct\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.414831 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.414796    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-164c36\"/\"kube-root-ca.crt\""
Nov 03 22:35:35.415057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.414838    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-164c36\"/\"simulator-w2-164c36-kserve-self-signed-certs\""
Nov 03 22:35:35.415057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.414896    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-164c36\"/\"openshift-service-ca.crt\""
Nov 03 22:35:35.415057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.414896    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-164c36\"/\"default-dockercfg-rb9gv\""
Nov 03 22:35:35.417235 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.417207    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"]
Nov 03 22:35:35.469415 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.469375    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-dshm\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.469737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.469444    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-tmp-dir\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.469737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.469500    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwct\" (UniqueName: \"kubernetes.io/projected/86774a7f-0cd5-43d6-9162-c9bf260453ed-kube-api-access-mdwct\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.469737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.469654    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-model-cache\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.469737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.469698    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86774a7f-0cd5-43d6-9162-c9bf260453ed-tls-certs\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.469982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.469760    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-home\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.470210 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.470185    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-home\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.470376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.470248    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-tmp-dir\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.470953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.470928    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-model-cache\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.487425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.487387    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-dshm\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.489388 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.489354    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86774a7f-0cd5-43d6-9162-c9bf260453ed-tls-certs\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.532568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.532536    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdwct\" (UniqueName: \"kubernetes.io/projected/86774a7f-0cd5-43d6-9162-c9bf260453ed-kube-api-access-mdwct\") pod \"simulator-w2-164c36-kserve-bc8cd59d8-4djvt\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") " pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:35.625153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:35.625064    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:36.004829 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:36.001856    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"]
Nov 03 22:35:36.541680 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:36.541629    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" event={"ID":"86774a7f-0cd5-43d6-9162-c9bf260453ed","Type":"ContainerStarted","Data":"eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9"}
Nov 03 22:35:36.542115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:36.541692    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" event={"ID":"86774a7f-0cd5-43d6-9162-c9bf260453ed","Type":"ContainerStarted","Data":"6150cbd07268f9e34e9686448b5922074858bebb29d59c0730a453ab48915577"}
Nov 03 22:35:36.625613 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:36.625580    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:36.663641 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:36.663596    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podStartSLOduration=1.6635838760000001 podStartE2EDuration="1.663583876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:35:36.661962696 +0000 UTC m=+760.102499955" watchObservedRunningTime="2026-11-03 22:35:36.663583876 +0000 UTC m=+760.104121134"
Nov 03 22:35:45.625846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:45.625807    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:45.638687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:45.638652    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:46.042389 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:35:46.042347    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:35:46.609430 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:46.609404    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:35:53.215545 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.215510    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"]
Nov 03 22:35:53.219638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.219618    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.323727 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.323660    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-1d31f8\"/\"openshift-service-ca.crt\""
Nov 03 22:35:53.323905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.323777    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-1d31f8\"/\"kube-root-ca.crt\""
Nov 03 22:35:53.323905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.323855    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d31f8\"/\"simulator-w3-1d31f8-kserve-self-signed-certs\""
Nov 03 22:35:53.323905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.323896    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-1d31f8\"/\"default-dockercfg-l54sw\""
Nov 03 22:35:53.325992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.325973    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"]
Nov 03 22:35:53.410787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.408001    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-dshm\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.410787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.408131    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-model-cache\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.410787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.408191    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-home\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.410787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.408360    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38135ba-2abe-4853-91a4-f4982fb3f727-tls-certs\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.410787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.408406    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5kld\" (UniqueName: \"kubernetes.io/projected/a38135ba-2abe-4853-91a4-f4982fb3f727-kube-api-access-v5kld\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.410787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.408590    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-tmp-dir\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511220 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511143    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-dshm\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511220 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511191    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-model-cache\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511220 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511214    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-home\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511244    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38135ba-2abe-4853-91a4-f4982fb3f727-tls-certs\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511263    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5kld\" (UniqueName: \"kubernetes.io/projected/a38135ba-2abe-4853-91a4-f4982fb3f727-kube-api-access-v5kld\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511297    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-tmp-dir\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511649 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511627    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-home\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511689    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-model-cache\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.511784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.511723    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-tmp-dir\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.513561 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.513534    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-dshm\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.513823 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.513805    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38135ba-2abe-4853-91a4-f4982fb3f727-tls-certs\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.571616 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.571580    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5kld\" (UniqueName: \"kubernetes.io/projected/a38135ba-2abe-4853-91a4-f4982fb3f727-kube-api-access-v5kld\") pod \"simulator-w3-1d31f8-kserve-665d69d48c-phq7s\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") " pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:53.832414 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:53.832329    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:35:54.210364 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:54.210339    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"]
Nov 03 22:35:54.212398 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:35:54.212369    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda38135ba_2abe_4853_91a4_f4982fb3f727.slice/crio-e1d6f403b4e028cec949aba6df9604b7906f2326b5794c06dc0c5cfc87322356 WatchSource:0}: Error finding container e1d6f403b4e028cec949aba6df9604b7906f2326b5794c06dc0c5cfc87322356: Status 404 returned error can't find the container with id e1d6f403b4e028cec949aba6df9604b7906f2326b5794c06dc0c5cfc87322356
Nov 03 22:35:54.651618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:54.651520    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" event={"ID":"a38135ba-2abe-4853-91a4-f4982fb3f727","Type":"ContainerStarted","Data":"902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c"}
Nov 03 22:35:54.651618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:54.651566    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" event={"ID":"a38135ba-2abe-4853-91a4-f4982fb3f727","Type":"ContainerStarted","Data":"e1d6f403b4e028cec949aba6df9604b7906f2326b5794c06dc0c5cfc87322356"}
Nov 03 22:35:54.769060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:54.769010    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podStartSLOduration=1.768995452 podStartE2EDuration="1.768995452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:35:54.767296462 +0000 UTC m=+778.207833722" watchObservedRunningTime="2026-11-03 22:35:54.768995452 +0000 UTC m=+778.209532710"
Nov 03 22:35:54.833355 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:35:54.833325    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:36:00.364983 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:00.364949    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:36:03.833393 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:03.833359    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:36:03.846285 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:03.846264    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:36:04.723195 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:04.723160    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:36:16.042179 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:36:16.042133    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:36:30.112877 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:30.112847    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:36:40.106818 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.106784    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"]
Nov 03 22:36:40.111012 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.110991    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.212447 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.212412    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-7540da\"/\"premium-w6-7540da-kserve-self-signed-certs\""
Nov 03 22:36:40.214708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.214685    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"]
Nov 03 22:36:40.291365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.287281    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-dshm\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.291365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.287459    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-model-cache\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.291365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.287490    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tls-certs\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.291365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.287543    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-home\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.291365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.287585    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tmp-dir\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.291365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.287639    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kknv\" (UniqueName: \"kubernetes.io/projected/445d4bb2-6baf-44ea-887d-4d0443b9fe01-kube-api-access-6kknv\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389240 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389130    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-dshm\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389240 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389235    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-model-cache\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389253    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tls-certs\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389290    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-home\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389316    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tmp-dir\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389361    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kknv\" (UniqueName: \"kubernetes.io/projected/445d4bb2-6baf-44ea-887d-4d0443b9fe01-kube-api-access-6kknv\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389753 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389649    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-model-cache\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389753 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389718    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-home\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.389857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.389774    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tmp-dir\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.391571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.391552    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-dshm\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.391865 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.391847    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tls-certs\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.450588 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.450560    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kknv\" (UniqueName: \"kubernetes.io/projected/445d4bb2-6baf-44ea-887d-4d0443b9fe01-kube-api-access-6kknv\") pod \"premium-w6-7540da-kserve-6bb5c64df7-7fxvp\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") " pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:40.723386 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:40.723357    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:41.096736 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:41.096701    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"]
Nov 03 22:36:41.099287 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:36:41.099262    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod445d4bb2_6baf_44ea_887d_4d0443b9fe01.slice/crio-33f73c2e920d83bb7452047d54c7b58ba5149a405ad8a5a88cb48c1071204281 WatchSource:0}: Error finding container 33f73c2e920d83bb7452047d54c7b58ba5149a405ad8a5a88cb48c1071204281: Status 404 returned error can't find the container with id 33f73c2e920d83bb7452047d54c7b58ba5149a405ad8a5a88cb48c1071204281
Nov 03 22:36:41.929283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:41.929249    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" event={"ID":"445d4bb2-6baf-44ea-887d-4d0443b9fe01","Type":"ContainerStarted","Data":"cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad"}
Nov 03 22:36:41.929752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:41.929291    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" event={"ID":"445d4bb2-6baf-44ea-887d-4d0443b9fe01","Type":"ContainerStarted","Data":"33f73c2e920d83bb7452047d54c7b58ba5149a405ad8a5a88cb48c1071204281"}
Nov 03 22:36:42.046088 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:42.046037    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podStartSLOduration=2.046023779 podStartE2EDuration="2.046023779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:36:42.044970918 +0000 UTC m=+825.485508178" watchObservedRunningTime="2026-11-03 22:36:42.046023779 +0000 UTC m=+825.486561036"
Nov 03 22:36:42.723440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:42.723409    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:46.043540 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:36:46.043506    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:36:50.723453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:50.723424    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:50.735908 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:50.735885    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:50.992011 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:50.991915    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:36:59.768842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:36:59.768806    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:37:15.600354 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.600314    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"]
Nov 03 22:37:15.604315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.604297    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.626388 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.626360    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-model-cache\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.626561 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.626444    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-dshm\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.626561 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.626521    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-home\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.626561 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.626555    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tmp-dir\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.626758 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.626713    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tls-certs\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.626836 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.626821    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw49c\" (UniqueName: \"kubernetes.io/projected/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-kube-api-access-hw49c\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.705694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.705630    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b64e2e\"/\"openshift-service-ca.crt\""
Nov 03 22:37:15.705694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.705635    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b64e2e\"/\"premium-w0-b64e2e-kserve-self-signed-certs\""
Nov 03 22:37:15.705694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.705635    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b64e2e\"/\"kube-root-ca.crt\""
Nov 03 22:37:15.705946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.705702    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b64e2e\"/\"default-dockercfg-ffw7x\""
Nov 03 22:37:15.707895 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.707874    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"]
Nov 03 22:37:15.728345 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728322    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw49c\" (UniqueName: \"kubernetes.io/projected/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-kube-api-access-hw49c\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.728492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728376    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-model-cache\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.728492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728433    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-dshm\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.728492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728452    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-home\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.728492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728466    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tmp-dir\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.728732 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728512    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tls-certs\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.728897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728876    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-model-cache\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.729007 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728940    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-home\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.729007 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.728971    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tmp-dir\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.730936 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.730917    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-dshm\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.731196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.731178    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tls-certs\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.792061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.792026    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw49c\" (UniqueName: \"kubernetes.io/projected/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-kube-api-access-hw49c\") pod \"premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") " pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:15.916925 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:15.916840    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:16.066502 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:37:16.066459    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:37:16.347925 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:16.347899    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"]
Nov 03 22:37:16.350053 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:37:16.350029    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ba930d7_6e52_45ac_a9c5_95e03546ee2d.slice/crio-03658dc7abf79ffd95aa762e28caec44383bb9afd32b3f7c279b27fda60e80f4 WatchSource:0}: Error finding container 03658dc7abf79ffd95aa762e28caec44383bb9afd32b3f7c279b27fda60e80f4: Status 404 returned error can't find the container with id 03658dc7abf79ffd95aa762e28caec44383bb9afd32b3f7c279b27fda60e80f4
Nov 03 22:37:17.126357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:17.126310    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" event={"ID":"2ba930d7-6e52-45ac-a9c5-95e03546ee2d","Type":"ContainerStarted","Data":"f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d"}
Nov 03 22:37:17.126357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:17.126360    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" event={"ID":"2ba930d7-6e52-45ac-a9c5-95e03546ee2d","Type":"ContainerStarted","Data":"03658dc7abf79ffd95aa762e28caec44383bb9afd32b3f7c279b27fda60e80f4"}
Nov 03 22:37:17.243445 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:17.243394    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podStartSLOduration=2.243381653 podStartE2EDuration="2.243381653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:37:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:37:17.242015737 +0000 UTC m=+860.682552994" watchObservedRunningTime="2026-11-03 22:37:17.243381653 +0000 UTC m=+860.683918910"
Nov 03 22:37:17.917455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:17.917421    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:22.180233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:22.180194    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:37:25.917859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:25.917829    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:25.929873 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:25.929852    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:26.187485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:26.187410    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:37:44.880004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:44.879969    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"]
Nov 03 22:37:44.883904 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:44.883887    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:44.985406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:44.985364    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-cbd26c\"/\"default-dockercfg-997r5\""
Nov 03 22:37:44.985406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:44.985364    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-cbd26c\"/\"openshift-service-ca.crt\""
Nov 03 22:37:44.985406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:44.985396    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-cbd26c\"/\"kube-root-ca.crt\""
Nov 03 22:37:44.985712 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:44.985486    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-cbd26c\"/\"premium-w5-cbd26c-kserve-self-signed-certs\""
Nov 03 22:37:44.988598 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:44.988576    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"]
Nov 03 22:37:45.042955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.032704    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6d8w\" (UniqueName: \"kubernetes.io/projected/0a7bd6ca-2661-493c-85d2-b2c490763360-kube-api-access-t6d8w\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.042955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.032785    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-dshm\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.042955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.032850    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-home\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.042955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.032875    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a7bd6ca-2661-493c-85d2-b2c490763360-tls-certs\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.042955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.032960    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-model-cache\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.042955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.033090    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-tmp-dir\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.134775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.134686    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-dshm\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.134775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.134739    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-home\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.134775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.134757    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a7bd6ca-2661-493c-85d2-b2c490763360-tls-certs\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.135049 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.134812    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-model-cache\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.135049 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.134898    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-tmp-dir\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.135049 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.134959    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6d8w\" (UniqueName: \"kubernetes.io/projected/0a7bd6ca-2661-493c-85d2-b2c490763360-kube-api-access-t6d8w\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.135208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.135117    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-home\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.135261 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.135221    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-model-cache\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.135354 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.135330    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-tmp-dir\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.137229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.137206    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-dshm\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.137790 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.137767    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a7bd6ca-2661-493c-85d2-b2c490763360-tls-certs\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.196567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.196531    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6d8w\" (UniqueName: \"kubernetes.io/projected/0a7bd6ca-2661-493c-85d2-b2c490763360-kube-api-access-t6d8w\") pod \"premium-w5-cbd26c-kserve-84d568866f-6z469\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") " pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.496510 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.496476    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:45.877166 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:45.877137    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"]
Nov 03 22:37:45.879198 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:37:45.879175    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a7bd6ca_2661_493c_85d2_b2c490763360.slice/crio-a8b478d662b484e94a061ebe63da242d8dd819a8b548600888c312fa30114623 WatchSource:0}: Error finding container a8b478d662b484e94a061ebe63da242d8dd819a8b548600888c312fa30114623: Status 404 returned error can't find the container with id a8b478d662b484e94a061ebe63da242d8dd819a8b548600888c312fa30114623
Nov 03 22:37:46.043338 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:37:46.043307    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:37:46.309917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:46.309880    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" event={"ID":"0a7bd6ca-2661-493c-85d2-b2c490763360","Type":"ContainerStarted","Data":"d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78"}
Nov 03 22:37:46.309917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:46.309919    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" event={"ID":"0a7bd6ca-2661-493c-85d2-b2c490763360","Type":"ContainerStarted","Data":"a8b478d662b484e94a061ebe63da242d8dd819a8b548600888c312fa30114623"}
Nov 03 22:37:46.427111 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:46.427053    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podStartSLOduration=2.427038449 podStartE2EDuration="2.427038449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:37:46.425551885 +0000 UTC m=+889.866089145" watchObservedRunningTime="2026-11-03 22:37:46.427038449 +0000 UTC m=+889.867575707"
Nov 03 22:37:46.496612 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:46.496578    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:54.363023 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:54.362990    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:37:55.496836 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:55.496789    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:55.509940 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:55.509903    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:56.393074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:56.393043    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:37:56.812580 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:56.812550    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:37:56.818494 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:37:56.818475    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:38:16.042529 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:38:16.042441    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:38:37.766829 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:38:37.766786    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:38:46.043971 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:38:46.043930    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:38:56.832294 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:38:56.832262    5647 generic.go:356] "Generic (PLEG): container finished" podID="b455b70d-d28f-48ac-a4ef-7a5c7392d06f" containerID="13d0ef43fdbf5b6e789e6db2210b174e145be9db1966a793ab544e4bf0a59197" exitCode=1
Nov 03 22:38:56.832764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:38:56.832306    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw" event={"ID":"b455b70d-d28f-48ac-a4ef-7a5c7392d06f","Type":"ContainerDied","Data":"13d0ef43fdbf5b6e789e6db2210b174e145be9db1966a793ab544e4bf0a59197"}
Nov 03 22:38:56.832764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:38:56.832658    5647 scope.go:122] "RemoveContainer" containerID="13d0ef43fdbf5b6e789e6db2210b174e145be9db1966a793ab544e4bf0a59197"
Nov 03 22:38:57.747010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:38:57.746981    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 03 22:38:57.841439 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:38:57.841406    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw" event={"ID":"b455b70d-d28f-48ac-a4ef-7a5c7392d06f","Type":"ContainerStarted","Data":"1eadbbf0c636b98785e3dd757b32e93397cdf5b8ee2a91700864ef44016a8c60"}
Nov 03 22:38:57.841873 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:38:57.841640    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:39:08.848826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:08.848795    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5d6776c6bc-p8lvw"
Nov 03 22:39:16.043561 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:39:16.043525    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:39:17.340062 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.340005    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"]
Nov 03 22:39:17.344343 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.344320    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445028    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9406813d-528c-4d15-8276-7d13f292cb13-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445238 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445077    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445238 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445190    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445358 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445274    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445358 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445339    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9406813d-528c-4d15-8276-7d13f292cb13-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445443    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445529 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445477    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46psr\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-kube-api-access-46psr\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445529 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445504    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.445628 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.445592    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.448218 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.448196    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"]
Nov 03 22:39:17.547263 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547234    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9406813d-528c-4d15-8276-7d13f292cb13-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.547439 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547273    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.547510 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547429    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.547565 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547518    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.547695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547584    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9406813d-528c-4d15-8276-7d13f292cb13-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.547695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547651    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.547830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547719    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.547991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547971    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.548057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.548022    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9406813d-528c-4d15-8276-7d13f292cb13-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.548119 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.548053    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.548192 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.547703    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46psr\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-kube-api-access-46psr\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.548251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.548224    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.548316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.548300    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.548547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.548524    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.550500 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.550472    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.550597 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.550514    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9406813d-528c-4d15-8276-7d13f292cb13-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.608399 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.608314    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.608557 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.608516    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46psr\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-kube-api-access-46psr\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.658529 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.658495    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:17.659011 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.658881    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"]
Nov 03 22:39:17.663262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.663248    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.727309 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.724890    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"]
Nov 03 22:39:17.751031 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751001    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.751197 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751063    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.751197 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751179    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvgsz\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-kube-api-access-qvgsz\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.751329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751214    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.751329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751305    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.751472 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751452    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.751536 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751506    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.751659 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751645    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.751723 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.751695    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853195 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853164    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853378 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853203    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853378 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853242    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853378 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853268    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853378 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853325    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvgsz\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-kube-api-access-qvgsz\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853608 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853419    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853608 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853567    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853749 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853639    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853749 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853723    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.853857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853745    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.854003 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.853976    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.854083 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.854022    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.854238 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.854221    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.854318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.854255    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.855735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.855716    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.856280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.856256    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.872445 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.872374    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:39:17.913564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.913529    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.913852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.913749    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvgsz\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-kube-api-access-qvgsz\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:17.990652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:17.990618    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:18.113809 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:18.111922    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"]
Nov 03 22:39:18.181262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:18.175285    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:18.181262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:18.175409    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:18.181262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:18.175459    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:18.556696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:18.556166    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"]
Nov 03 22:39:18.638171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:18.635691    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:18.638171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:18.635807    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:18.638171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:18.635876    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:19.021743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:19.021701    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" event={"ID":"99dbf9ad-841d-4052-ae15-b99f25bb68ca","Type":"ContainerStarted","Data":"fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3"}
Nov 03 22:39:19.023806 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:19.023773    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" event={"ID":"99dbf9ad-841d-4052-ae15-b99f25bb68ca","Type":"ContainerStarted","Data":"7ac1203b927b5d1c7fa99439a96da88e528a3a4f61c74c730d58f010a3db5e5a"}
Nov 03 22:39:19.026271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:19.026249    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" event={"ID":"9406813d-528c-4d15-8276-7d13f292cb13","Type":"ContainerStarted","Data":"b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9"}
Nov 03 22:39:19.026399 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:19.026296    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" event={"ID":"9406813d-528c-4d15-8276-7d13f292cb13","Type":"ContainerStarted","Data":"99af68fa87886f505996636343dd62a87a9ea88fb62c4fbdfcfdda86120a29d7"}
Nov 03 22:39:19.168696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:19.166234    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podStartSLOduration=2.166215147 podStartE2EDuration="2.166215147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:39:19.155593158 +0000 UTC m=+982.596130418" watchObservedRunningTime="2026-11-03 22:39:19.166215147 +0000 UTC m=+982.606752407"
Nov 03 22:39:19.665132 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:19.665091    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:19.668468 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:19.668430    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:39:19.668761 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:19.668728    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:39:20.037721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:20.023630    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:20.037721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:20.023782    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 03 22:39:20.037721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:20.023829    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 03 22:39:20.544989 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:20.543649    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:39:20.661881 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:20.661843    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:39:20.662154 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:20.662128    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:39:20.994180 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:20.994144    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 03 22:39:20.994840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:20.994799    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 03 22:39:21.659239 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:21.659206    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:39:21.659451 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:21.659264    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:39:21.991694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:21.991062    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 03 22:39:21.991694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:21.991122    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 03 22:39:22.659108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:22.659076    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:39:22.659536 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:22.659127    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:39:22.995579 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:22.995101    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 03 22:39:22.995579 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:22.995168    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 03 22:39:23.659793 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:23.659758    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:39:23.660217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:23.659810    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:39:23.991037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:23.990999    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 03 22:39:23.991316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:23.991290    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 03 22:39:24.658877 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:24.658842    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 03 22:39:24.659061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:24.658898    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 03 22:39:24.992005 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:24.991968    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 03 22:39:24.992624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:24.992593    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 03 22:39:25.992518 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:25.992482    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 03 22:39:25.992958 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:25.992548    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 03 22:39:26.700881 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.700840    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:39:26.701087 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.700914    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:39:26.717141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.717105    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:26.718265 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.718236    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:26.756017 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.755988    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 22:39:26.873616 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.873561    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podStartSLOduration=9.8735479 podStartE2EDuration="9.8735479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:39:19.282435525 +0000 UTC m=+982.722972784" watchObservedRunningTime="2026-11-03 22:39:26.8735479 +0000 UTC m=+990.314085225"
Nov 03 22:39:26.996633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.996605    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:26.997196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.997015    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:26.997854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:26.997825    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:39:31.754857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.754818    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"]
Nov 03 22:39:31.759321 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.759297    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.811323 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.811284    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-b64e2e-openshift-default-dockercfg-g4cxs\""
Nov 03 22:39:31.823969 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.823945    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.824099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.823974    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.824099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.823999    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.824170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.824121    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.824170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.824142    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.824170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.824161    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a292b26-d09e-4f9b-8efd-53d8016b0475-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.824262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.824220    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.824262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.824256    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.824324 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.824281    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv8fj\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-kube-api-access-zv8fj\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.864050 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.864022    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"]
Nov 03 22:39:31.925255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925217    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925257    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925378    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a292b26-d09e-4f9b-8efd-53d8016b0475-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925436    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925468    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925511    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv8fj\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-kube-api-access-zv8fj\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925753 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925567    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925753 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925741    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925773    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925791    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.925842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925809    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.926001 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.925854    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.926168 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.926145    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.926248 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.926174    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a292b26-d09e-4f9b-8efd-53d8016b0475-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.927573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.927551    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.928254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.928232    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.986406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.986372    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv8fj\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-kube-api-access-zv8fj\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:31.986619 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:31.986597    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:32.015356 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.015285    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"]
Nov 03 22:39:32.020426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.020410    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.072645 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.072614    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-7afb1c-openshift-default-dockercfg-nlmnh\""
Nov 03 22:39:32.072828 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.072759    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:32.083886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.083857    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"]
Nov 03 22:39:32.129174 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129136    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.129174 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129177    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dfa5234-0fb8-4b64-b263-313228dc237e-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.129418 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129249    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.129418 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129292    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjmr9\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-kube-api-access-gjmr9\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.129418 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129400    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.129582 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129437    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.129582 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129522    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.129582 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129557    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.129780 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.129644    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.230628 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.230597    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.230852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.230644    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.230852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.230719    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.230852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.230746    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.230864    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.230942    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.230969    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dfa5234-0fb8-4b64-b263-313228dc237e-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.231009    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.231034    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjmr9\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-kube-api-access-gjmr9\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.231075    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.231363    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231443 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.231419    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231499 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.231470    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.231764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.231741    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dfa5234-0fb8-4b64-b263-313228dc237e-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.233399 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.233372    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.233611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.233595    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.291319 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.291263    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.291435 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.291416    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjmr9\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-kube-api-access-gjmr9\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.334887 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.334855    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:32.524287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.524257    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"]
Nov 03 22:39:32.525892 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:39:32.525867    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a292b26_d09e_4f9b_8efd_53d8016b0475.slice/crio-bb06c03ac6746d0160bf0c49541acec2916ab64e92975a537d89e1f19fe4e59a WatchSource:0}: Error finding container bb06c03ac6746d0160bf0c49541acec2916ab64e92975a537d89e1f19fe4e59a: Status 404 returned error can't find the container with id bb06c03ac6746d0160bf0c49541acec2916ab64e92975a537d89e1f19fe4e59a
Nov 03 22:39:32.528403 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.528373    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:32.528466 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.528440    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:32.528507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.528467    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:32.717553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.717521    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"]
Nov 03 22:39:32.717768 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:39:32.717746    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfa5234_0fb8_4b64_b263_313228dc237e.slice/crio-6493a275d9bb6db4d80f29ddf7c0b6146393de8299489f08c578d257c8342d14 WatchSource:0}: Error finding container 6493a275d9bb6db4d80f29ddf7c0b6146393de8299489f08c578d257c8342d14: Status 404 returned error can't find the container with id 6493a275d9bb6db4d80f29ddf7c0b6146393de8299489f08c578d257c8342d14
Nov 03 22:39:32.720352 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.720318    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:32.720446 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.720388    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:32.720446 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:32.720430    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:39:33.215205 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.215164    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" event={"ID":"5a292b26-d09e-4f9b-8efd-53d8016b0475","Type":"ContainerStarted","Data":"8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be"}
Nov 03 22:39:33.215720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.215230    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" event={"ID":"5a292b26-d09e-4f9b-8efd-53d8016b0475","Type":"ContainerStarted","Data":"bb06c03ac6746d0160bf0c49541acec2916ab64e92975a537d89e1f19fe4e59a"}
Nov 03 22:39:33.217236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.217207    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" event={"ID":"8dfa5234-0fb8-4b64-b263-313228dc237e","Type":"ContainerStarted","Data":"d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397"}
Nov 03 22:39:33.217372 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.217255    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" event={"ID":"8dfa5234-0fb8-4b64-b263-313228dc237e","Type":"ContainerStarted","Data":"6493a275d9bb6db4d80f29ddf7c0b6146393de8299489f08c578d257c8342d14"}
Nov 03 22:39:33.335431 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.335290    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:33.337160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.337134    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 03 22:39:33.337287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.337181    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 03 22:39:33.337787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.337747    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podStartSLOduration=2.337733889 podStartE2EDuration="2.337733889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:39:33.334830293 +0000 UTC m=+996.775367553" watchObservedRunningTime="2026-11-03 22:39:33.337733889 +0000 UTC m=+996.778271148"
Nov 03 22:39:33.464117 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:33.464066    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podStartSLOduration=2.464053683 podStartE2EDuration="2.464053683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:39:33.460579051 +0000 UTC m=+996.901116337" watchObservedRunningTime="2026-11-03 22:39:33.464053683 +0000 UTC m=+996.904590990"
Nov 03 22:39:34.077696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:34.073117    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:34.077696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:34.076787    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:34.077696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:34.076846    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:34.335256 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:34.335178    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 03 22:39:34.335722 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:34.335656    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 03 22:39:35.105280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:35.102028    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:35.105280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:35.102086    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:35.335740 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:35.335703    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 03 22:39:35.336127 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:35.335746    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 03 22:39:36.078696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:36.075059    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:36.078696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:36.075123    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:36.336072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:36.335985    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 03 22:39:36.336072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:36.336035    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 03 22:39:37.075284 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:37.075251    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:37.075586 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:37.075556    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:37.335704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:37.335593    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 03 22:39:37.335984 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:37.335954    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 03 22:39:38.081104 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:38.081063    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:38.081735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:38.081700    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:38.348256 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:38.348157    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 03 22:39:38.348519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:38.348488    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 03 22:39:38.606754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:38.605764    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:39:38.722590 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:38.722543    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:39:39.056752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:39.038029    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:39:39.082169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:39.082130    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 03 22:39:39.082848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:39.082812    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 03 22:39:39.116702 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:39.115826    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:39:39.335453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:39.335357    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 03 22:39:39.335453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:39.335426    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 03 22:39:40.224027 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:40.223988    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:40.224461 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:40.224437    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:40.225760 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:40.225736    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:39:40.935989 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:40.935961    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:40.936297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:40.936273    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:40.937594 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:40.937568    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 22:39:46.042747 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:39:46.042711    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:39:55.380565 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:39:55.380524    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:40:16.044048 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:40:16.044007    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:40:20.444362 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:20.444323    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:40:33.256405 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.256375    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"]
Nov 03 22:40:33.261002 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.260981    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.359752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.359712    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.359949 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.359818    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.359949 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.359853    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.360074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.359968    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.360074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.360000    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.360074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.360030    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4l8w\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-kube-api-access-w4l8w\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.360074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.360068    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.360294 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.360136    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.360294 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.360171    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.366953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.366926    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"]
Nov 03 22:40:33.461683 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461636    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.461857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461708    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.461857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461766    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.461857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461815    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.461857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461835    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461904    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461924    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461942    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4l8w\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-kube-api-access-w4l8w\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.461966    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.462066    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-credential-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.462143    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-certs\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462390 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.462368    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-data\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462430 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.462420    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-socket\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.462974 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.462951    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istiod-ca-cert\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.464339 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.464319    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-podinfo\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.464428 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.464353    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-envoy\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.522429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.522354    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-token\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.522570 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.522518    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4l8w\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-kube-api-access-w4l8w\") pod \"e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") " pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:33.574337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:33.574302    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:34.006047 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:34.006019    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"]
Nov 03 22:40:34.007835 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:40:34.007808    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf80ffbd_f32e_4b7c_93a6_64ea5a661b2c.slice/crio-b092b42beb7f10e0646cc11b18215211d151d3fabcadda093e7bc123fd807826 WatchSource:0}: Error finding container b092b42beb7f10e0646cc11b18215211d151d3fabcadda093e7bc123fd807826: Status 404 returned error can't find the container with id b092b42beb7f10e0646cc11b18215211d151d3fabcadda093e7bc123fd807826
Nov 03 22:40:34.010282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:34.010252    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:40:34.010364 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:34.010315    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:40:34.010364 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:34.010344    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:40:34.695738 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:34.695697    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" event={"ID":"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c","Type":"ContainerStarted","Data":"cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735"}
Nov 03 22:40:34.696121 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:34.695768    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" event={"ID":"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c","Type":"ContainerStarted","Data":"b092b42beb7f10e0646cc11b18215211d151d3fabcadda093e7bc123fd807826"}
Nov 03 22:40:34.861190 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:34.861134    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podStartSLOduration=1.861115006 podStartE2EDuration="1.861115006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:40:34.860052791 +0000 UTC m=+1058.300590076" watchObservedRunningTime="2026-11-03 22:40:34.861115006 +0000 UTC m=+1058.301652269"
Nov 03 22:40:35.591693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:35.591026    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:35.598099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:35.598067    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 03 22:40:35.598304 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:35.598122    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 03 22:40:36.590318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:36.585100    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 03 22:40:36.590318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:36.585177    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 03 22:40:37.575123 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:37.575080    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 03 22:40:37.575419 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:37.575387    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 03 22:40:38.588182 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:38.588146    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 03 22:40:38.588798 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:38.588754    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 03 22:40:39.578062 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:39.578023    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 03 22:40:39.578468 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:39.578438    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 03 22:40:40.575339 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:40.575309    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 03 22:40:40.575735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:40.575353    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 03 22:40:41.579445 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:41.579410    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:41.579887 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:41.579718    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:41.580339 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:40:41.580318    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:40:46.043449 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:40:46.043400    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:41:02.108963 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.108931    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"]
Nov 03 22:41:02.114385 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.114361    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223143 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223106    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223151    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223293    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223396 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223335    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223396 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223357    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0753c62f-7777-49c6-8090-bc2093554768-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223510 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223495    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkhd7\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-kube-api-access-qkhd7\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223566 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223554    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0753c62f-7777-49c6-8090-bc2093554768-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223617 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223590    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.223847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.223822    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.254902 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.254870    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"]
Nov 03 22:41:02.289694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.289634    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:41:02.324880 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.324849    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.324922    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.324953    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.325017    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.325052    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.325079    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0753c62f-7777-49c6-8090-bc2093554768-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.325143    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkhd7\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-kube-api-access-qkhd7\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.325194    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0753c62f-7777-49c6-8090-bc2093554768-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325439 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.325300    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325529 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.325509    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.325786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.325754    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.326505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.326189    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.326644 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.326339    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.326780 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.326374    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0753c62f-7777-49c6-8090-bc2093554768-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.328360 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.328340    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.328645 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.328625    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0753c62f-7777-49c6-8090-bc2093554768-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.385583 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.385508    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.385779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.385717    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkhd7\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-kube-api-access-qkhd7\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.427384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.427351    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:02.860014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.859971    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"]
Nov 03 22:41:02.923015 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.922970    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:41:02.923597 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.923572    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:41:02.951886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:02.951839    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:41:03.018541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:03.017760    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" event={"ID":"0753c62f-7777-49c6-8090-bc2093554768","Type":"ContainerStarted","Data":"4e7050959b81556b8c535c969ba5c4079aed42a507e1255dc6727ada512402eb"}
Nov 03 22:41:04.038501 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:04.038447    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" event={"ID":"0753c62f-7777-49c6-8090-bc2093554768","Type":"ContainerStarted","Data":"e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37"}
Nov 03 22:41:04.263073 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:04.263021    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podStartSLOduration=2.263008913 podStartE2EDuration="2.263008913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:41:04.211902207 +0000 UTC m=+1087.652439465" watchObservedRunningTime="2026-11-03 22:41:04.263008913 +0000 UTC m=+1087.703546171"
Nov 03 22:41:04.428178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:04.428093    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:04.429800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:04.429774    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 03 22:41:04.429939 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:04.429825    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 03 22:41:05.428965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:05.428920    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 03 22:41:05.430868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:05.430831    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 03 22:41:06.428243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:06.428210    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 03 22:41:06.428426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:06.428277    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 03 22:41:07.428288 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:07.428253    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 03 22:41:07.428724 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:07.428310    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 03 22:41:08.428006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:08.427967    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 03 22:41:08.428192 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:08.428042    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 03 22:41:09.427873 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:09.427836    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 03 22:41:09.428286 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:09.427898    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 03 22:41:10.432306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:10.432272    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:10.432822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:10.432638    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:10.433476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:10.433459    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:41:12.268590 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.268555    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"]
Nov 03 22:41:12.272762 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.272732    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.356304 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.356260    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"]
Nov 03 22:41:12.376650 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.376624    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b64e2e\"/\"distinct-2-w0-b64e2e-kserve-self-signed-certs\""
Nov 03 22:41:12.406350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.406314    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-tmp-dir\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.406523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.406423    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7nlz\" (UniqueName: \"kubernetes.io/projected/44a7b927-3900-4a1f-9270-b72dd3239865-kube-api-access-k7nlz\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.406523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.406449    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-model-cache\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.406621 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.406540    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-home\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.406621 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.406605    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a7b927-3900-4a1f-9270-b72dd3239865-tls-certs\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.406810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.406793    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-dshm\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.508002 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.507952    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-tmp-dir\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.508199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.508070    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7nlz\" (UniqueName: \"kubernetes.io/projected/44a7b927-3900-4a1f-9270-b72dd3239865-kube-api-access-k7nlz\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.508199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.508111    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-model-cache\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.508199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.508149    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-home\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.508383 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.508224    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a7b927-3900-4a1f-9270-b72dd3239865-tls-certs\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.508383 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.508337    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-dshm\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.508485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.508403    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-tmp-dir\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.509035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.508981    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-home\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.509035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.508998    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-model-cache\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.512996 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.512972    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-dshm\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.513141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.513111    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a7b927-3900-4a1f-9270-b72dd3239865-tls-certs\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.572613 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.572522    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7nlz\" (UniqueName: \"kubernetes.io/projected/44a7b927-3900-4a1f-9270-b72dd3239865-kube-api-access-k7nlz\") pod \"distinct-2-w0-b64e2e-kserve-7459659c68-x2277\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") " pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:12.585592 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:12.585566    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:13.001257 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:13.001232    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"]
Nov 03 22:41:13.002824 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:41:13.002800    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44a7b927_3900_4a1f_9270_b72dd3239865.slice/crio-8d3e27382118d1956cb8a8da9462fa6b92644c6521308a40874f124b58a38d05 WatchSource:0}: Error finding container 8d3e27382118d1956cb8a8da9462fa6b92644c6521308a40874f124b58a38d05: Status 404 returned error can't find the container with id 8d3e27382118d1956cb8a8da9462fa6b92644c6521308a40874f124b58a38d05
Nov 03 22:41:13.095893 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:13.095858    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" event={"ID":"44a7b927-3900-4a1f-9270-b72dd3239865","Type":"ContainerStarted","Data":"b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7"}
Nov 03 22:41:13.096037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:13.095909    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" event={"ID":"44a7b927-3900-4a1f-9270-b72dd3239865","Type":"ContainerStarted","Data":"8d3e27382118d1956cb8a8da9462fa6b92644c6521308a40874f124b58a38d05"}
Nov 03 22:41:13.213474 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:13.213426    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podStartSLOduration=1.213412102 podStartE2EDuration="1.213412102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:41:13.211105561 +0000 UTC m=+1096.651642819" watchObservedRunningTime="2026-11-03 22:41:13.213412102 +0000 UTC m=+1096.653949360"
Nov 03 22:41:13.585920 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:13.585869    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:16.044010 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:41:16.043970    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:41:22.585759 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:22.585715    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:22.598416 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:22.598389    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:23.216448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:23.216389    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:41:43.964442 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:43.964401    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:41:46.043656 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:41:46.043618    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:41:49.581245 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.581208    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"]
Nov 03 22:41:49.585358 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.585339    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.660152 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.660116    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"]
Nov 03 22:41:49.689027 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.688996    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-164c36\"/\"distinct-2-w2-164c36-kserve-self-signed-certs\""
Nov 03 22:41:49.691526 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.691497    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-home\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.691653 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.691613    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-dshm\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.691727 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.691708    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-tmp-dir\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.691782 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.691729    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7892a50d-3c45-481c-8c05-2a70be4d71b5-tls-certs\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.691866 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.691848    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc6zz\" (UniqueName: \"kubernetes.io/projected/7892a50d-3c45-481c-8c05-2a70be4d71b5-kube-api-access-lc6zz\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.691957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.691942    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-model-cache\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.799610    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-model-cache\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.799775    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-home\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.799874    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-dshm\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.799984    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-tmp-dir\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.800013    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7892a50d-3c45-481c-8c05-2a70be4d71b5-tls-certs\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.800076    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc6zz\" (UniqueName: \"kubernetes.io/projected/7892a50d-3c45-481c-8c05-2a70be4d71b5-kube-api-access-lc6zz\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.800945    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-model-cache\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.801696    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-home\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.803558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.803160    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-tmp-dir\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.808243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.808217    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7892a50d-3c45-481c-8c05-2a70be4d71b5-tls-certs\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.808456 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.808409    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-dshm\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.861978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.861909    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc6zz\" (UniqueName: \"kubernetes.io/projected/7892a50d-3c45-481c-8c05-2a70be4d71b5-kube-api-access-lc6zz\") pod \"distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") " pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:49.898899 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:49.898863    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:50.337370 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:50.337337    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"]
Nov 03 22:41:50.339030 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:41:50.338992    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7892a50d_3c45_481c_8c05_2a70be4d71b5.slice/crio-6da66ad19c1cc3c60ed2319e8d2ba7c3952026aeaf67310229b74207208e636d WatchSource:0}: Error finding container 6da66ad19c1cc3c60ed2319e8d2ba7c3952026aeaf67310229b74207208e636d: Status 404 returned error can't find the container with id 6da66ad19c1cc3c60ed2319e8d2ba7c3952026aeaf67310229b74207208e636d
Nov 03 22:41:50.392945 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:50.392920    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" event={"ID":"7892a50d-3c45-481c-8c05-2a70be4d71b5","Type":"ContainerStarted","Data":"6da66ad19c1cc3c60ed2319e8d2ba7c3952026aeaf67310229b74207208e636d"}
Nov 03 22:41:51.400943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:51.400906    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" event={"ID":"7892a50d-3c45-481c-8c05-2a70be4d71b5","Type":"ContainerStarted","Data":"bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5"}
Nov 03 22:41:51.517273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:51.517218    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podStartSLOduration=2.51720314 podStartE2EDuration="2.51720314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:41:51.516493005 +0000 UTC m=+1134.957030267" watchObservedRunningTime="2026-11-03 22:41:51.51720314 +0000 UTC m=+1134.957740398"
Nov 03 22:41:51.899637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:51.899598    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:59.899365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:59.899332    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:41:59.911986 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:41:59.911959    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:42:00.492797 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:00.492765    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:42:16.043929 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:42:16.043886    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:42:24.994377 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:24.994341    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:42:30.174554 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.174512    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"]
Nov 03 22:42:30.179297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.179269    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.244634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.244595    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"]
Nov 03 22:42:30.246652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.246619    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-32449d\"/\"openshift-service-ca.crt\""
Nov 03 22:42:30.248625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.248579    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-32449d\"/\"default-dockercfg-86zlr\""
Nov 03 22:42:30.254264 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.254231    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-dshm\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.254401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.254386    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-home\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.254475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.254426    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvqpv\" (UniqueName: \"kubernetes.io/projected/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-kube-api-access-mvqpv\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.254590 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.254570    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tls-certs\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.254652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.254615    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tmp-dir\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.254834 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.254809    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-model-cache\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.278862 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.278834    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-32449d\"/\"kube-root-ca.crt\""
Nov 03 22:42:30.282508 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.282491    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-32449d\"/\"distinct-2-w4-32449d-kserve-self-signed-certs\""
Nov 03 22:42:30.358698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.356756    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tls-certs\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.358698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.356823    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tmp-dir\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.358698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.356927    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-model-cache\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.358698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.357004    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-dshm\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.358698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.357104    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-home\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.358698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.357140    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvqpv\" (UniqueName: \"kubernetes.io/projected/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-kube-api-access-mvqpv\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.381705 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.361205    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-model-cache\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.383567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.374144    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tmp-dir\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.384726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.384701    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-home\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.404707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.402412    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tls-certs\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.404707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.403091    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-dshm\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.419251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.419214    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvqpv\" (UniqueName: \"kubernetes.io/projected/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-kube-api-access-mvqpv\") pod \"distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") " pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.492941 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.492900    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:30.904005 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:30.903977    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"]
Nov 03 22:42:30.905920 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:42:30.905891    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0315c1df_36d9_41a1_a3f9_581bb5aee4dd.slice/crio-cefd7bc6adcc83535f60f2a740a99be54c76aa33017bac6d372015b6c4abb423 WatchSource:0}: Error finding container cefd7bc6adcc83535f60f2a740a99be54c76aa33017bac6d372015b6c4abb423: Status 404 returned error can't find the container with id cefd7bc6adcc83535f60f2a740a99be54c76aa33017bac6d372015b6c4abb423
Nov 03 22:42:31.724941 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:31.724908    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" event={"ID":"0315c1df-36d9-41a1-a3f9-581bb5aee4dd","Type":"ContainerStarted","Data":"f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6"}
Nov 03 22:42:31.724941 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:31.724952    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" event={"ID":"0315c1df-36d9-41a1-a3f9-581bb5aee4dd","Type":"ContainerStarted","Data":"cefd7bc6adcc83535f60f2a740a99be54c76aa33017bac6d372015b6c4abb423"}
Nov 03 22:42:31.843982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:31.843924    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podStartSLOduration=1.8439105420000002 podStartE2EDuration="1.843910542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:42:31.842496226 +0000 UTC m=+1175.283033485" watchObservedRunningTime="2026-11-03 22:42:31.843910542 +0000 UTC m=+1175.284447800"
Nov 03 22:42:32.493758 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:32.493728    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:40.494100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:40.494062    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:40.507557 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:40.507520    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:40.852173 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:40.852086    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:42:46.043444 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:42:46.043396    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:42:49.213317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.213284    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"]
Nov 03 22:42:49.217521 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.217501    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.276153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.276121    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"]
Nov 03 22:42:49.320916 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.320885    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-cbd26c\"/\"distinct-2-w5-cbd26c-kserve-self-signed-certs\""
Nov 03 22:42:49.399428 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.399385    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-dshm\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.399634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.399441    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tls-certs\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.399634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.399529    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c46dp\" (UniqueName: \"kubernetes.io/projected/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-kube-api-access-c46dp\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.399634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.399590    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-model-cache\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.399634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.399610    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tmp-dir\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.399897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.399714    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-home\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.500917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.500821    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-model-cache\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.500917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.500859    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tmp-dir\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.501155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.500944    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-home\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.501155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.501104    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-dshm\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.501155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.501132    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tls-certs\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.501313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.501220    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c46dp\" (UniqueName: \"kubernetes.io/projected/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-kube-api-access-c46dp\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.501313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.501247    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-model-cache\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.501410 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.501318    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tmp-dir\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.501410 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.501366    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-home\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.503543 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.503522    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-dshm\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.503923 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.503903    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tls-certs\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.562234 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.562201    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c46dp\" (UniqueName: \"kubernetes.io/projected/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-kube-api-access-c46dp\") pod \"distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") " pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:49.831874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:49.831779    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:50.212165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:50.212123    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"]
Nov 03 22:42:50.214257 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:42:50.214217    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c82ed1b_3e6e_48eb_986b_0285d98d7eb2.slice/crio-8b32326cf7fcc132a105a8f580cdea6367a83fd7253838904736bff0ced75887 WatchSource:0}: Error finding container 8b32326cf7fcc132a105a8f580cdea6367a83fd7253838904736bff0ced75887: Status 404 returned error can't find the container with id 8b32326cf7fcc132a105a8f580cdea6367a83fd7253838904736bff0ced75887
Nov 03 22:42:50.927735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:50.926945    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" event={"ID":"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2","Type":"ContainerStarted","Data":"38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d"}
Nov 03 22:42:50.927735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:50.927004    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" event={"ID":"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2","Type":"ContainerStarted","Data":"8b32326cf7fcc132a105a8f580cdea6367a83fd7253838904736bff0ced75887"}
Nov 03 22:42:51.047048 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:51.046982    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podStartSLOduration=2.046964348 podStartE2EDuration="2.046964348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:42:51.046564434 +0000 UTC m=+1194.487101688" watchObservedRunningTime="2026-11-03 22:42:51.046964348 +0000 UTC m=+1194.487501609"
Nov 03 22:42:51.832921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:51.832855    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:56.859152 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:56.859122    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:42:56.863990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:56.863970    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:42:59.832557 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:59.832515    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:59.845513 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:59.845478    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:42:59.996828 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:42:59.996798    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:43:03.043554 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.043516    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"]
Nov 03 22:43:03.048757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.048735    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.101773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.101739    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.101946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.101800    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.101946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.101856    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vjf8\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-kube-api-access-7vjf8\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.102020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.101942    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.102072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.102049    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.102109 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.102081    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.102167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.102156    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.102202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.102189    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.102235 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.102217    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204034 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.203992    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204240 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204047    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204240 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204073    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vjf8\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-kube-api-access-7vjf8\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204240 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204147    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204240 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204205    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204240 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204229    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204282    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204314    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204356    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204531    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.204691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204581    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-credential-socket\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.205000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.204957    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-data\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.205103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.205032    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-certs\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.205198 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.205173    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istiod-ca-cert\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.206949 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.206924    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-envoy\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.207650 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.207627    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-podinfo\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.258036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.257996    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"]
Nov 03 22:43:03.265898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.265865    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-token\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.268346 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.268313    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vjf8\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-kube-api-access-7vjf8\") pod \"e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") " pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.363347 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.363257    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:03.418513 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.418477    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"]
Nov 03 22:43:03.426383 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.426349    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.486953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.486802    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"]
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.508477    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbmqm\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-kube-api-access-gbmqm\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.508518    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.508568    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.508600    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.508977    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.509036    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.509072    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.509134    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.515721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.509158    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.610352 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.610305    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.610549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.610374    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.610549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.610406    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.610549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.610461    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.610549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.610482    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.610549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.610525    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbmqm\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-kube-api-access-gbmqm\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.610844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.610573    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.611021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.610994    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.611105 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.611086    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.611211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.611187    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.611364 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.611233    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.611544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.611521    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.611842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.611818    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.612180 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.612156    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.613076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.613055    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.613425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.613382    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.671582 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.671550    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.671782 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.671713    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbmqm\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-kube-api-access-gbmqm\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.755929 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.755890    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:03.839262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.839230    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"]
Nov 03 22:43:03.844720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.844660    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:03.844834 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.844779    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:03.844834 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:03.844824    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:04.014993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.014958    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" event={"ID":"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec","Type":"ContainerStarted","Data":"9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866"}
Nov 03 22:43:04.015221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.015021    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" event={"ID":"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec","Type":"ContainerStarted","Data":"090afd9266ec04563daac5ded7455b15a5c789b1fafeba77e200ac471647687b"}
Nov 03 22:43:04.136359 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.136232    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podStartSLOduration=2.136213234 podStartE2EDuration="2.136213234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:04.135536316 +0000 UTC m=+1207.576073576" watchObservedRunningTime="2026-11-03 22:43:04.136213234 +0000 UTC m=+1207.576750494"
Nov 03 22:43:04.144019 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:43:04.143984    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0b763c_b2c0_410b_bcf2_b7728d61bf55.slice/crio-feaa75f0f383fcc20916ad9973b4fddaeff149e614273a58af7de73510562799 WatchSource:0}: Error finding container feaa75f0f383fcc20916ad9973b4fddaeff149e614273a58af7de73510562799: Status 404 returned error can't find the container with id feaa75f0f383fcc20916ad9973b4fddaeff149e614273a58af7de73510562799
Nov 03 22:43:04.144384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.144361    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"]
Nov 03 22:43:04.146899 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.146857    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:04.146991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.146946    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:04.147039 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.146988    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:04.363523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.363492    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:04.365065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.365036    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:04.365190 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:04.365088    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:05.034816 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:05.034781    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" event={"ID":"7f0b763c-b2c0-410b-bcf2-b7728d61bf55","Type":"ContainerStarted","Data":"e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf"}
Nov 03 22:43:05.035029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:05.034842    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" event={"ID":"7f0b763c-b2c0-410b-bcf2-b7728d61bf55","Type":"ContainerStarted","Data":"feaa75f0f383fcc20916ad9973b4fddaeff149e614273a58af7de73510562799"}
Nov 03 22:43:05.387744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:05.384017    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:05.387744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:05.384076    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:05.757061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:05.757028    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:05.758201 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:05.758178    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:43:05.758256 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:05.758243    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:43:06.364119 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:06.364081    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:06.364385 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:06.364351    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:06.756852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:06.756812    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:43:06.757262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:06.756865    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:43:07.364581 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:07.364542    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:07.364778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:07.364595    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:07.756337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:07.756299    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:43:07.756521 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:07.756352    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:43:07.797694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:07.796898    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podStartSLOduration=4.796877385 podStartE2EDuration="4.796877385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:05.159224656 +0000 UTC m=+1208.599761915" watchObservedRunningTime="2026-11-03 22:43:07.796877385 +0000 UTC m=+1211.237414644"
Nov 03 22:43:07.806692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:07.800641    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:43:08.364585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:08.364543    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:08.364857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:08.364615    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:08.757049 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:08.757017    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:43:08.757205 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:08.757066    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:43:09.366171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:09.366135    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:09.366851 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:09.366799    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:09.757157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:09.757111    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:43:09.757476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:09.757429    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:43:09.982887 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:09.981781    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:10.225697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.223945    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:10.366097 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.366061    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:10.366380 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.366351    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:10.423352 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.422912    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:10.493756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.490766    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:10.513697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.509464    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:10.603693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.602763    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:10.735708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.734408    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:43:10.760215 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.760120    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:43:10.760475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.760446    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:43:10.849696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.848822    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:10.849696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.848991    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:43:10.984844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.984790    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:10.986125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:10.986097    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:43:11.007696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.001839    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:11.212226 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.210173    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Nov 03 22:43:11.234376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.232369    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:43:11.234376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.232857    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:43:11.368436 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.368398    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:11.370976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.370934    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:11.546757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.517809    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:11.546757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.518133    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:43:11.546757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.518331    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/health\": context deadline exceeded"
Nov 03 22:43:11.546757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.518436    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:43:11.546757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.518492    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:43:11.635700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.631568    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:11.663812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.656871    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:43:11.773813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.751623    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:43:11.781919 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.753034    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": context deadline exceeded"
Nov 03 22:43:11.782405 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.782377    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:43:11.933753 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.894937    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 03 22:43:11.933753 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.895009    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 03 22:43:11.933753 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.895940    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:11.933753 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:11.897441    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:43:12.042695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.026631    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:43:12.231707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.225760    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:12.231707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.094455    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:43:12.250759 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.094212    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:12.250759 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.243943    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:43:12.302214 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.302185    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:43:12.306640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.306617    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:43:12.319847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.319259    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:43:12.325810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.320327    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:12.340567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.340539    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:43:12.363407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.357316    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:43:12.363407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.361560    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:43:12.373688 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.373254    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 03 22:43:12.373688 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.373320    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 03 22:43:12.378329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.378303    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:43:12.394414 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:12.394373    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:43:13.167691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:13.167308    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:13.169704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:13.168022    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:13.169704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:13.169323    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:43:13.446810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:13.446725    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:13.447993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:13.447918    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:13.449173 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:13.449149    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:43:16.042353 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:43:16.042315    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:43:19.231068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.231036    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"]
Nov 03 22:43:19.235371 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.235352    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.308245 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.308208    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.308546 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.308496    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45k4t\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-kube-api-access-45k4t\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.308653 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.308572    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.308653 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.308606    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.308876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.308855    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.309001 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.308896    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.309001 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.308947    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64740797-9960-4f13-9a00-a3c5e8eb11c7-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.309001 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.308973    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.309143 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.309017    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.333522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.333489    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"]
Nov 03 22:43:19.410327 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410296    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45k4t\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-kube-api-access-45k4t\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410350    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410372    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410488    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410523    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410655 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410570    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64740797-9960-4f13-9a00-a3c5e8eb11c7-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410655 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410596    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410655 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410639    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410726    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.410946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410920    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-credential-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.411013 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.410955    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-certs\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.411072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.411016    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-data\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.411195 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.411167    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-socket\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.411325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.411304    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64740797-9960-4f13-9a00-a3c5e8eb11c7-istiod-ca-cert\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.412926 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.412906    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-envoy\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.413530 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.413511    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-podinfo\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.470912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.470883    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-token\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.471122 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.471100    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45k4t\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-kube-api-access-45k4t\") pod \"e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") " pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.550565 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.550481    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:19.984092 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.984067    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"]
Nov 03 22:43:19.985283 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:43:19.985256    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64740797_9960_4f13_9a00_a3c5e8eb11c7.slice/crio-feacea3b4c379d768901afd2267a45f140ead2e6ad68b3c77c96c6a08eccbf78 WatchSource:0}: Error finding container feacea3b4c379d768901afd2267a45f140ead2e6ad68b3c77c96c6a08eccbf78: Status 404 returned error can't find the container with id feacea3b4c379d768901afd2267a45f140ead2e6ad68b3c77c96c6a08eccbf78
Nov 03 22:43:19.988021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.987990    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:19.988106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.988052    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:19.988106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:19.988099    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:20.428409 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:20.428368    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" event={"ID":"64740797-9960-4f13-9a00-a3c5e8eb11c7","Type":"ContainerStarted","Data":"4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1"}
Nov 03 22:43:20.428897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:20.428426    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" event={"ID":"64740797-9960-4f13-9a00-a3c5e8eb11c7","Type":"ContainerStarted","Data":"feacea3b4c379d768901afd2267a45f140ead2e6ad68b3c77c96c6a08eccbf78"}
Nov 03 22:43:20.560074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:20.560040    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:21.551958 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:21.551926    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:43:21.552335 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:21.551977    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:43:22.551797 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:22.551761    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:43:22.551980 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:22.551822    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:43:23.550911 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:23.550872    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:43:23.551121 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:23.550935    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:43:24.551422 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:24.551385    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:43:24.551875 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:24.551438    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:43:25.551751 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:25.551722    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:43:25.552145 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:25.551770    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:43:26.550878 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:26.550838    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 03 22:43:26.551060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:26.550898    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 03 22:43:27.555596 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:27.555566    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:27.556084 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:27.555979    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:27.556730 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:27.556710    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:43:27.773904 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:27.773851    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podStartSLOduration=8.773838327 podStartE2EDuration="8.773838327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:20.559544264 +0000 UTC m=+1224.000081526" watchObservedRunningTime="2026-11-03 22:43:27.773838327 +0000 UTC m=+1231.214375588"
Nov 03 22:43:33.660054 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.660019    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"]
Nov 03 22:43:33.665310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.665293    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.725136 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.725094    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"]
Nov 03 22:43:33.733379 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.733348    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.733546 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.733460    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.733546 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.733509    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.733546 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.733530    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83443efc-b020-45fb-a722-0e7eb4e5c719-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.733868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.733849    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.734037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.734022    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.734087 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.734051    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.734127 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.734117    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4wmd\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-kube-api-access-w4wmd\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.734169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.734145    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835088    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835304 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835166    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835304 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835219    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835304 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835247    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83443efc-b020-45fb-a722-0e7eb4e5c719-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835304 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835294    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835511    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835594 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835544    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835657 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835612    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4wmd\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-kube-api-access-w4wmd\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835729 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835645    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-certs\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835814    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-credential-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835907 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835653    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.835996 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.835976    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-data\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.836252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.836226    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-socket\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.836324 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.836252    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83443efc-b020-45fb-a722-0e7eb4e5c719-istiod-ca-cert\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.837706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.837662    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-envoy\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.838272 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.838250    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-podinfo\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.896663 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.896625    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-token\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.896834 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.896662    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4wmd\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-kube-api-access-w4wmd\") pod \"e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") " pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:33.980250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:33.980212    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:34.419714 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.419687    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"]
Nov 03 22:43:34.420865 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:43:34.420835    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83443efc_b020_45fb_a722_0e7eb4e5c719.slice/crio-71ee101a4f9642a97dea194f83985d22247e3f17d9a28da5975111c0997d7cb8 WatchSource:0}: Error finding container 71ee101a4f9642a97dea194f83985d22247e3f17d9a28da5975111c0997d7cb8: Status 404 returned error can't find the container with id 71ee101a4f9642a97dea194f83985d22247e3f17d9a28da5975111c0997d7cb8
Nov 03 22:43:34.423320 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.423287    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:34.423402 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.423351    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:34.423402 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.423380    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:34.523563 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.523516    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" event={"ID":"83443efc-b020-45fb-a722-0e7eb4e5c719","Type":"ContainerStarted","Data":"30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07"}
Nov 03 22:43:34.523743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.523581    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" event={"ID":"83443efc-b020-45fb-a722-0e7eb4e5c719","Type":"ContainerStarted","Data":"71ee101a4f9642a97dea194f83985d22247e3f17d9a28da5975111c0997d7cb8"}
Nov 03 22:43:34.645735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.645645    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podStartSLOduration=1.645629214 podStartE2EDuration="1.645629214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:34.643813924 +0000 UTC m=+1238.084351185" watchObservedRunningTime="2026-11-03 22:43:34.645629214 +0000 UTC m=+1238.086166473"
Nov 03 22:43:34.849288 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.849247    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"]
Nov 03 22:43:34.853803 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.853783    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.905907 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.905867    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-cbd26c-openshift-default-dockercfg-5rskr\""
Nov 03 22:43:34.915566 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.915532    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"]
Nov 03 22:43:34.954801 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.954750    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.955109 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.955081    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.955279 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.955260    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.955449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.955432    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.955626 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.955611    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.957954 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.957923    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.958083 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.957992    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.958083 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.958034    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.958201 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.958082    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrkrx\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-kube-api-access-zrkrx\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:34.994691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:34.994639    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:35.059624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.059593    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.059871 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.059631    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.059871 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.059708    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.059881    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.059940    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.059991    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.060040    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.060074    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrkrx\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-kube-api-access-zrkrx\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.060089    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.060254    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.060257    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060574 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.060547    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.060606    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.060756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.060663    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.062353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.062327    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.063065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.063044    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.120333 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.120260    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.120466 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.120435    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrkrx\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-kube-api-access-zrkrx\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.168297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.168264    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:35.322710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.321919    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"]
Nov 03 22:43:35.328388 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.328359    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.369846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.369777    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w5mc\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-kube-api-access-8w5mc\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.369846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.369842    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.370113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.369915    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.370113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.370020    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.370113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.370066    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.370279 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.370134    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.370279 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.370162    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.370279 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.370236    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.370458 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.370360    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.490708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.479658    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w5mc\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-kube-api-access-8w5mc\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.490708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.479741    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.490708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.489497    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.479806    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.517261    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.517313    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.517384    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.517413    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.517492    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.517641    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.518688    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.518946    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.519184    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.531010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.526557    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.555840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.555740    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.556449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.556420    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w5mc\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-kube-api-access-8w5mc\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.564105 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.564046    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"]
Nov 03 22:43:35.568011 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.567526    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.601146 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.601102    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.649376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.649292    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:35.699260 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.699235    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"]
Nov 03 22:43:35.702403 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:43:35.702373    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358d8eaa_554b_4fd5_aaa1_39c81432b9c1.slice/crio-c768a1c8252c9e20efbc167a0e4b85a3e6ec33dfc122f8c014382e83eca5cc29 WatchSource:0}: Error finding container c768a1c8252c9e20efbc167a0e4b85a3e6ec33dfc122f8c014382e83eca5cc29: Status 404 returned error can't find the container with id c768a1c8252c9e20efbc167a0e4b85a3e6ec33dfc122f8c014382e83eca5cc29
Nov 03 22:43:35.705516 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.705475    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:35.705608 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.705567    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:35.705648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.705607    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:35.998835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.998800    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:35.999400 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:35.999366    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:36.063757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.063726    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"]
Nov 03 22:43:36.068786 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:43:36.068758    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b8beb3_d6b3_449c_8ec2_d7a5fec73aec.slice/crio-f88d11de488f437b59e8c6b8dae7bde51c6f326910b6c0e9cb8cb467278b40a7 WatchSource:0}: Error finding container f88d11de488f437b59e8c6b8dae7bde51c6f326910b6c0e9cb8cb467278b40a7: Status 404 returned error can't find the container with id f88d11de488f437b59e8c6b8dae7bde51c6f326910b6c0e9cb8cb467278b40a7
Nov 03 22:43:36.083725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.079656    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:36.083725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.079774    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:36.083725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.079817    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:43:36.565681 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.565628    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" event={"ID":"358d8eaa-554b-4fd5-aaa1-39c81432b9c1","Type":"ContainerStarted","Data":"161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1"}
Nov 03 22:43:36.565995 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.565970    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" event={"ID":"358d8eaa-554b-4fd5-aaa1-39c81432b9c1","Type":"ContainerStarted","Data":"c768a1c8252c9e20efbc167a0e4b85a3e6ec33dfc122f8c014382e83eca5cc29"}
Nov 03 22:43:36.567859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.567839    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" event={"ID":"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec","Type":"ContainerStarted","Data":"64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57"}
Nov 03 22:43:36.567981 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.567970    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" event={"ID":"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec","Type":"ContainerStarted","Data":"f88d11de488f437b59e8c6b8dae7bde51c6f326910b6c0e9cb8cb467278b40a7"}
Nov 03 22:43:36.649807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.649763    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:36.840275 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.840130    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podStartSLOduration=1.8401124690000001 podStartE2EDuration="1.840112469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:36.837879428 +0000 UTC m=+1240.278416687" watchObservedRunningTime="2026-11-03 22:43:36.840112469 +0000 UTC m=+1240.280649731"
Nov 03 22:43:36.843217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.843160    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podStartSLOduration=2.84314696 podStartE2EDuration="2.84314696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:43:36.708039126 +0000 UTC m=+1240.148576389" watchObservedRunningTime="2026-11-03 22:43:36.84314696 +0000 UTC m=+1240.283684218"
Nov 03 22:43:36.989695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.987931    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:36.989695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:36.987995    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:37.168880 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:37.168798    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:37.191710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:37.170634    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:37.191710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:37.170743    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:37.652191 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:37.652147    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:37.652406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:37.652212    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:38.006697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:38.006611    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:38.006697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:38.006687    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:38.169490 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:38.169450    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:38.169883 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:38.169518    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:38.650248 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:38.650217    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:38.650589 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:38.650547    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:38.985696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:38.985004    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:38.985696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:38.985060    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:39.169631 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:39.169591    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:39.170054 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:39.169651    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:39.650577 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:39.650536    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:39.650786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:39.650593    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:39.982695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:39.981384    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:39.982695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:39.981448    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:40.169252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:40.169214    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:40.169429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:40.169272    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:40.650378 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:40.650331    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:40.651002 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:40.650958    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:40.995057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:40.995020    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 03 22:43:40.995369 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:40.995338    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 03 22:43:41.169284 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:41.169248    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:41.169464 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:41.169302    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:41.650192 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:41.650153    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:41.650351 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:41.650216    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:42.169367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:42.169335    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 03 22:43:42.169957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:42.169925    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 03 22:43:42.381085 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:42.381055    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:42.382815 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:42.382793    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:42.403232 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:42.403204    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 22:43:42.650248 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:42.650217    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 03 22:43:42.650427 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:42.650272    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 03 22:43:43.516162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:43.516133    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:43.517038 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:43.517012    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:43.518475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:43.518453    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:43:43.858762 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:43.858142    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:43.859396 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:43.859348    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:43.860478 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:43.860459    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:43:46.221156 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:43:46.221120    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:43:46.925715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:46.917981    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:47.205882 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.205839    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:47.303798 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.303750    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:47.326182 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.325793    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Nov 03 22:43:47.344765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.343983    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:47.345536 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.345508    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:47.364812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.363314    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:47.364812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.363454    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:47.380213 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.378749    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:47.380213 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.379147    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:43:47.403492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:47.403451    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:48.206328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:48.206270    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:43:48.207750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:48.207729    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:43:48.235576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:43:48.235540    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:44:04.153885 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.153847    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"]
Nov 03 22:44:04.160541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.160513    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.240604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.240565    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.240810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.240788    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.240897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.240885    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.240960 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.240917    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.241037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.241023    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.241103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.241057    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.241103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.241081    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.241207 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.241139    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.241263 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.241233    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw6qk\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-kube-api-access-sw6qk\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.269350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.269318    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"]
Nov 03 22:44:04.342263 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342231    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342274    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342313    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342337    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342381    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw6qk\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-kube-api-access-sw6qk\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342435    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342595    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342767    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342781    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342784    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342956 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342863    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.342956 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.342928    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.343194 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.343176    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.343628 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.343587    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.344794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.344766    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.345522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.345501    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.405126 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.405042    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.405271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.405137    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw6qk\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-kube-api-access-sw6qk\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.477343 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.477307    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:04.979495 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.979466    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"]
Nov 03 22:44:04.981033 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:44:04.981005    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3881c5b_b563_45f1_9609_8ba6cc8fb68a.slice/crio-42840f3ff90c41f62f7189ab64b2d54c60b7667c515933cd1bc070eb3d50548c WatchSource:0}: Error finding container 42840f3ff90c41f62f7189ab64b2d54c60b7667c515933cd1bc070eb3d50548c: Status 404 returned error can't find the container with id 42840f3ff90c41f62f7189ab64b2d54c60b7667c515933cd1bc070eb3d50548c
Nov 03 22:44:04.984042 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.984007    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:04.984159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.984076    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:04.984159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:04.984118    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:05.274814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:05.274718    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" event={"ID":"b3881c5b-b563-45f1-9609-8ba6cc8fb68a","Type":"ContainerStarted","Data":"cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb"}
Nov 03 22:44:05.274814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:05.274771    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" event={"ID":"b3881c5b-b563-45f1-9609-8ba6cc8fb68a","Type":"ContainerStarted","Data":"42840f3ff90c41f62f7189ab64b2d54c60b7667c515933cd1bc070eb3d50548c"}
Nov 03 22:44:05.396738 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:05.396615    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podStartSLOduration=1.3965982750000001 podStartE2EDuration="1.396598275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:05.39522035 +0000 UTC m=+1268.835757612" watchObservedRunningTime="2026-11-03 22:44:05.396598275 +0000 UTC m=+1268.837135536"
Nov 03 22:44:05.478277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:05.478242    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:06.495694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:06.495249    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:44:06.495694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:06.495315    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:44:07.495695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:07.492805    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:44:07.495695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:07.492872    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:44:08.497757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:08.490149    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:44:08.497757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:08.490226    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:44:09.496869 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:09.496830    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:44:09.497068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:09.496904    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:44:10.485090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:10.485048    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:44:10.487176 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:10.487142    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:44:11.478788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:11.478748    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:44:11.478968 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:11.478828    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:44:12.482407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:12.482378    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:12.482857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:12.482840    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:12.483379 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:12.483359    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:44:16.043919 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:44:16.043884    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:44:19.323348 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:19.323312    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:44:22.252706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.239800    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Nov 03 22:44:22.334721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.314761    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:22.334721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.315484    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:22.334721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.330757    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:22.366700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.347968    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:22.366700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.364217    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:44:22.379733 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.367839    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:22.389189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.384757    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:22.389189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.385155    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:22.444732 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:22.420199    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:44:23.240126 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.240071    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:23.265770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.265729    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:44:23.333348 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.333308    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:23.351456 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.351415    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:44:23.376021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.353089    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:44:23.376021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.353158    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:44:23.376021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.354783    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:23.376021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.371892    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:44:23.376021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.372976    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:23.376021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.374133    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:44:23.376021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.375005    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:44:23.379790 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.377635    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:44:23.380839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.380799    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:44:23.409909 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.404981    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:44:23.409909 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.406070    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:44:23.409909 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.407357    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:44:23.409909 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.408434    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:44:23.409909 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.408488    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:44:23.447573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.447536    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:44:23.467910 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.467862    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:44:23.686158 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:23.686050    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.025507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.025474    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:44:24.094030 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.093991    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:44:24.094345 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.094301    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.213202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.213169    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:44:24.288754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.278864    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.340932 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.337542    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.428704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.395941    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.454862 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.399453    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:44:24.455136 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.401651    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.455261 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.409112    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.455386 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.449083    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.484811 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.449172    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:24.930694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:24.929731    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:25.054929 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:25.054515    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:44:25.054929 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:25.054585    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:44:25.054929 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:25.054620    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:44:25.054929 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:25.054652    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:44:25.116176 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:25.116148    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:44:25.168713 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:25.167996    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:44:25.201243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:25.201156    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:44:25.209585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:25.208554    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:44:38.686426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:38.686391    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"]
Nov 03 22:44:38.686952 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:38.686597    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" containerID="cri-o://d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f" gracePeriod=30
Nov 03 22:44:38.738031 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:38.737995    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"]
Nov 03 22:44:38.738287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:38.738261    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy" containerID="cri-o://c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf" gracePeriod=30
Nov 03 22:44:38.742173 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:38.742145    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"]
Nov 03 22:44:38.742459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:38.742432    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" containerID="cri-o://a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20" gracePeriod=30
Nov 03 22:44:42.116773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:42.116737    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:44:42.117182 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:42.116806    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:44:43.149422 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:43.149388    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:44:43.149868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:43.149441    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:44:43.715532 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:43.715502    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Nov 03 22:44:43.715621 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:43.715560    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Nov 03 22:44:43.996123 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:43.995778    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:44:44.046506 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.046469    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-credential-socket\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.046826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.046549    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-envoy\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.046826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.046594    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-kube-api-access-fpgw9\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-kube-api-access-fpgw9\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.046826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.046745    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38825124-f803-4234-ad30-349fb215b0b1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38825124-f803-4234-ad30-349fb215b0b1-istiod-ca-cert\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.046826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.046786    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-data\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.047165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.046924    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/38825124-f803-4234-ad30-349fb215b0b1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38825124-f803-4234-ad30-349fb215b0b1-istio-podinfo\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.047165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.047053    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-socket\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.047165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.047142    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-certs\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.047393 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.047174    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-istio-token\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-istio-token\") pod \"38825124-f803-4234-ad30-349fb215b0b1\" (UID: \"38825124-f803-4234-ad30-349fb215b0b1\") "
Nov 03 22:44:44.048706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.048565    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38825124-f803-4234-ad30-349fb215b0b1-istiod-ca-cert" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:44:44.049368 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.048994    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-data" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.049368 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.049320    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-socket" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.050005 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.049378    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-certs" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.050005 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.049432    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-credential-socket" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.053622 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.053594    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-envoy" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.056782 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.056755    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38825124-f803-4234-ad30-349fb215b0b1-istio-podinfo" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:44:44.072840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.072808    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-istio-token" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:44:44.073795 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.073745    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-kube-api-access-fpgw9" pod "38825124-f803-4234-ad30-349fb215b0b1" (UID: "38825124-f803-4234-ad30-349fb215b0b1"). InnerVolumeSpecName "kube-api-access-fpgw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:44:44.086537 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.086512    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:44:44.101874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.101850    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:44:44.148444 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148409    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-socket\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.148444 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148449    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-podinfo\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.148716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148475    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-kube-api-access-46h5n\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-kube-api-access-46h5n\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.148716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148507    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-certs\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.148716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148534    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-credential-socket\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.148716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148583    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-envoy\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.148716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148616    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-envoy\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.148979 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148730    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-token\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-token\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.148979 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148756    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-credential-socket\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.148979 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148778    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-socket\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.148979 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148781    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-socket" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.148979 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148816    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-podinfo\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.149255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.148990    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-certs\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.149255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149025    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-data\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.149255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149027    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-credential-socket" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.149255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149089    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-token\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-token\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.149255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149144    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e56eae5f-6e03-493d-a288-c29f37f06b3a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e56eae5f-6e03-493d-a288-c29f37f06b3a-istiod-ca-cert\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.149255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149170    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-credential-socket" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.149255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149185    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-kube-api-access-fggkg\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-kube-api-access-fggkg\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.150208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149352    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e4508fb-4d66-4503-826a-155a08bcc9df-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e4508fb-4d66-4503-826a-155a08bcc9df-istiod-ca-cert\") pod \"6e4508fb-4d66-4503-826a-155a08bcc9df\" (UID: \"6e4508fb-4d66-4503-826a-155a08bcc9df\") "
Nov 03 22:44:44.150208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149403    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-data\") pod \"e56eae5f-6e03-493d-a288-c29f37f06b3a\" (UID: \"e56eae5f-6e03-493d-a288-c29f37f06b3a\") "
Nov 03 22:44:44.150208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.149501    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-certs" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150366    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-socket" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150440    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150465    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150468    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e4508fb-4d66-4503-826a-155a08bcc9df-istiod-ca-cert" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150482    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150497    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150511    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150525    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150540    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpgw9\" (UniqueName: \"kubernetes.io/projected/38825124-f803-4234-ad30-349fb215b0b1-kube-api-access-fpgw9\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150554    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150568    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150583    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38825124-f803-4234-ad30-349fb215b0b1-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150596    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38825124-f803-4234-ad30-349fb215b0b1-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150611    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150626    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38825124-f803-4234-ad30-349fb215b0b1-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150565    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-certs" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.150872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150770    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-data" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.152055 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.150910    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-data" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.152055 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.151071    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e56eae5f-6e03-493d-a288-c29f37f06b3a-istiod-ca-cert" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:44:44.152055 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.151490    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-envoy" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.152231 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.152094    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-token" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:44:44.152231 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.152131    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-token" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:44:44.152409 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.152385    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-kube-api-access-fggkg" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "kube-api-access-fggkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:44:44.152477 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.152405    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-envoy" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:44:44.152646 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.152621    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-podinfo" pod "e56eae5f-6e03-493d-a288-c29f37f06b3a" (UID: "e56eae5f-6e03-493d-a288-c29f37f06b3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:44:44.152887 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.152869    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-kube-api-access-46h5n" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "kube-api-access-46h5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:44:44.153460 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.153443    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-podinfo" pod "6e4508fb-4d66-4503-826a-155a08bcc9df" (UID: "6e4508fb-4d66-4503-826a-155a08bcc9df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:44:44.251976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251893    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.251976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251920    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.251976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251932    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.251976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251941    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e56eae5f-6e03-493d-a288-c29f37f06b3a-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.251976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251951    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fggkg\" (UniqueName: \"kubernetes.io/projected/e56eae5f-6e03-493d-a288-c29f37f06b3a-kube-api-access-fggkg\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.251976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251960    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e4508fb-4d66-4503-826a-155a08bcc9df-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.251976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251968    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.251976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251977    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.252337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251986    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46h5n\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-kube-api-access-46h5n\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.252337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.251995    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.252337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.252003    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e56eae5f-6e03-493d-a288-c29f37f06b3a-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.252337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.252010    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.252337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.252018    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e4508fb-4d66-4503-826a-155a08bcc9df-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.252337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.252037    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e4508fb-4d66-4503-826a-155a08bcc9df-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:44:44.540714 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.540599    5647 generic.go:356] "Generic (PLEG): container finished" podID="38825124-f803-4234-ad30-349fb215b0b1" containerID="c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf" exitCode=0
Nov 03 22:44:44.540714 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.540649    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"
Nov 03 22:44:44.540714 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.540700    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" event={"ID":"38825124-f803-4234-ad30-349fb215b0b1","Type":"ContainerDied","Data":"c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf"}
Nov 03 22:44:44.541002 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.540764    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq" event={"ID":"38825124-f803-4234-ad30-349fb215b0b1","Type":"ContainerDied","Data":"594425c6bf53b8cf1d575e893488493511e8eb79d44420abbca4b622f6f5b074"}
Nov 03 22:44:44.541002 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.540792    5647 scope.go:122] "RemoveContainer" containerID="c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf"
Nov 03 22:44:44.542081 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.542059    5647 generic.go:356] "Generic (PLEG): container finished" podID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerID="d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f" exitCode=0
Nov 03 22:44:44.542170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.542108    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"
Nov 03 22:44:44.542170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.542113    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" event={"ID":"e56eae5f-6e03-493d-a288-c29f37f06b3a","Type":"ContainerDied","Data":"d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f"}
Nov 03 22:44:44.542170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.542163    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj" event={"ID":"e56eae5f-6e03-493d-a288-c29f37f06b3a","Type":"ContainerDied","Data":"35266ea9e51626532e9ea865c85f7c18f310d9803c1356877871f3b6baff60f4"}
Nov 03 22:44:44.543820 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.543794    5647 generic.go:356] "Generic (PLEG): container finished" podID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerID="a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20" exitCode=0
Nov 03 22:44:44.543928 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.543839    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"
Nov 03 22:44:44.543928 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.543872    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" event={"ID":"6e4508fb-4d66-4503-826a-155a08bcc9df","Type":"ContainerDied","Data":"a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20"}
Nov 03 22:44:44.544043 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.543932    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p" event={"ID":"6e4508fb-4d66-4503-826a-155a08bcc9df","Type":"ContainerDied","Data":"787b8509048d0c4ef3812dea825d48b8da31fcbff097759a00c9115a98023df0"}
Nov 03 22:44:44.553882 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.553865    5647 scope.go:122] "RemoveContainer" containerID="c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf"
Nov 03 22:44:44.554216 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:44:44.554198    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf\": container with ID starting with c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf not found: ID does not exist" containerID="c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf"
Nov 03 22:44:44.554284 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.554224    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf"} err="failed to get container status \"c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf\": rpc error: code = NotFound desc = could not find container \"c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf\": container with ID starting with c9cba1c0dd68a91becab7df99f9dae9c1ce9358dc90a83313588a57f08068acf not found: ID does not exist"
Nov 03 22:44:44.554284 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.554241    5647 scope.go:122] "RemoveContainer" containerID="d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f"
Nov 03 22:44:44.564141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.564119    5647 scope.go:122] "RemoveContainer" containerID="d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f"
Nov 03 22:44:44.564404 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:44:44.564387    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f\": container with ID starting with d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f not found: ID does not exist" containerID="d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f"
Nov 03 22:44:44.564476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.564416    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f"} err="failed to get container status \"d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f\": rpc error: code = NotFound desc = could not find container \"d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f\": container with ID starting with d56044a2f9960c88887c6a1be033bb5171d3e23fdbe133d6451e35cdedc0611f not found: ID does not exist"
Nov 03 22:44:44.564476 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.564442    5647 scope.go:122] "RemoveContainer" containerID="a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20"
Nov 03 22:44:44.574264 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.574244    5647 scope.go:122] "RemoveContainer" containerID="a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20"
Nov 03 22:44:44.574573 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:44:44.574553    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20\": container with ID starting with a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20 not found: ID does not exist" containerID="a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20"
Nov 03 22:44:44.574643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.574580    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20"} err="failed to get container status \"a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20\": rpc error: code = NotFound desc = could not find container \"a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20\": container with ID starting with a43d6f69ba021e7a3421dfe2be04d1589c5f7d0c60acf378c01f8280b3199b20 not found: ID does not exist"
Nov 03 22:44:44.706071 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.706041    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"]
Nov 03 22:44:44.710228 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.710203    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj"]
Nov 03 22:44:44.819651 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.819561    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"]
Nov 03 22:44:44.823131 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.823107    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p"]
Nov 03 22:44:44.931843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.931804    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"]
Nov 03 22:44:44.936433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:44.936410    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq"]
Nov 03 22:44:46.043764 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:44:46.043730    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:44:46.507620 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.507585    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"]
Nov 03 22:44:46.517194 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.517159    5647 cpu_manager.go:414] "containerMap: removing container" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy"
Nov 03 22:44:46.517401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.517384    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy"
Nov 03 22:44:46.517625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.517610    5647 cpu_manager.go:414] "containerMap: removing container" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy"
Nov 03 22:44:46.517807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.517791    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy"
Nov 03 22:44:46.517910 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.517900    5647 cpu_manager.go:414] "containerMap: removing container" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy"
Nov 03 22:44:46.517981 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.517971    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy"
Nov 03 22:44:46.518235 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.518223    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" containerName="istio-proxy"
Nov 03 22:44:46.518329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.518320    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="38825124-f803-4234-ad30-349fb215b0b1" containerName="istio-proxy"
Nov 03 22:44:46.518424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.518415    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" containerName="istio-proxy"
Nov 03 22:44:46.560700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.548051    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.581569 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.581524    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"]
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610163    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-podinfo\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610294    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-certs\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610323    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-data\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610469    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa54680-0cbb-48f4-b08d-f391d5c94158-istiod-ca-cert\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610517    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pftnk\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-kube-api-access-pftnk\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610620    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-envoy\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610662    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-credential-socket\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610717    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-socket\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.610938    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-token\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.618876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.611546    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-fc129ccf-gw-openshift-default-dockercfg-fjr2t\""
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795072    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-certs\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795111    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-data\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795226    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa54680-0cbb-48f4-b08d-f391d5c94158-istiod-ca-cert\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795251    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pftnk\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-kube-api-access-pftnk\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795322    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-envoy\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795343    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-credential-socket\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795378    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-socket\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795503    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-token\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.795732    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-podinfo\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.796521    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-data\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.796768    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-certs\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.797415    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa54680-0cbb-48f4-b08d-f391d5c94158-istiod-ca-cert\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.797652    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-socket\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.808921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.797892    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-credential-socket\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.811626 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.811600    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38825124-f803-4234-ad30-349fb215b0b1" path="/var/lib/kubelet/pods/38825124-f803-4234-ad30-349fb215b0b1/volumes"
Nov 03 22:44:46.812162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.812120    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e4508fb-4d66-4503-826a-155a08bcc9df" path="/var/lib/kubelet/pods/6e4508fb-4d66-4503-826a-155a08bcc9df/volumes"
Nov 03 22:44:46.812605 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.812522    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e56eae5f-6e03-493d-a288-c29f37f06b3a" path="/var/lib/kubelet/pods/e56eae5f-6e03-493d-a288-c29f37f06b3a/volumes"
Nov 03 22:44:46.813250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.813194    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-envoy\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.813757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.813734    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-podinfo\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.862737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.862713    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-token\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.862873 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.862778    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pftnk\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-kube-api-access-pftnk\") pod \"e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") " pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:46.908590 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:46.908568    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:47.399929 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:47.399893    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"]
Nov 03 22:44:47.401152 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:44:47.401127    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa54680_0cbb_48f4_b08d_f391d5c94158.slice/crio-c4a590cc8cee75296ddcf012e75df45ee4cba80fef3003d511fb292ad3283a9d WatchSource:0}: Error finding container c4a590cc8cee75296ddcf012e75df45ee4cba80fef3003d511fb292ad3283a9d: Status 404 returned error can't find the container with id c4a590cc8cee75296ddcf012e75df45ee4cba80fef3003d511fb292ad3283a9d
Nov 03 22:44:47.403617 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:47.403587    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:47.403689 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:47.403649    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:47.403733 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:47.403712    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:47.593843 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:47.593802    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" event={"ID":"daa54680-0cbb-48f4-b08d-f391d5c94158","Type":"ContainerStarted","Data":"038aa33460d32934ec9d304003206b8cfc410fb39ef37c460c44a75dad379fe6"}
Nov 03 22:44:47.594043 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:47.593863    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" event={"ID":"daa54680-0cbb-48f4-b08d-f391d5c94158","Type":"ContainerStarted","Data":"c4a590cc8cee75296ddcf012e75df45ee4cba80fef3003d511fb292ad3283a9d"}
Nov 03 22:44:47.716103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:47.716048    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" podStartSLOduration=1.7160339690000002 podStartE2EDuration="1.716033969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:47.712809811 +0000 UTC m=+1311.153347070" watchObservedRunningTime="2026-11-03 22:44:47.716033969 +0000 UTC m=+1311.156571229"
Nov 03 22:44:47.918729 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:47.913810    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:48.915187 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:48.915158    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:49.478961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.478928    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"]
Nov 03 22:44:49.483315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.483297    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.582106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.582078    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"]
Nov 03 22:44:49.609566 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.609539    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:49.610715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.610660    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:44:49.645290 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645259    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzrj7\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-kube-api-access-vzrj7\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.645477 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645312    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.645477 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645380    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.645568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645499    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.645568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645532    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.645568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645564    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.645754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645730    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.645789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645768    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.645828 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.645821    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.747307 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.747220    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.747307 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.747257    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.747539 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.747396    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.747659 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.747639    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzrj7\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-kube-api-access-vzrj7\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.747750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.747716    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.747809 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.747760    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.747915 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.747892    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.747981 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.747962    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.748050 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.748009    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.748202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.748185    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-data\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.748486 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.748441    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.748579 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.748513    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istiod-ca-cert\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.748707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.748652    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-credential-socket\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.748790 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.748770    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-certs\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.749975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.749952    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-envoy\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.750979 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.750955    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-podinfo\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.808774 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.808746    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzrj7\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-kube-api-access-vzrj7\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:49.808917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:49.808785    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-token\") pod \"e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") " pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:50.097521 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:50.097421    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:50.533365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:50.533331    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"]
Nov 03 22:44:50.534285 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:44:50.534260    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3b44d3_d34b_4837_bd03_ade32eb4a473.slice/crio-f3a06798b232a524dbf3af6d04b4ca475b0bb3fea7eefb61baaf283d1274e092 WatchSource:0}: Error finding container f3a06798b232a524dbf3af6d04b4ca475b0bb3fea7eefb61baaf283d1274e092: Status 404 returned error can't find the container with id f3a06798b232a524dbf3af6d04b4ca475b0bb3fea7eefb61baaf283d1274e092
Nov 03 22:44:50.537189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:50.537158    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:50.537287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:50.537227    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:50.537287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:50.537259    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:44:50.620327 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:50.620301    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" event={"ID":"2b3b44d3-d34b-4837-bd03-ade32eb4a473","Type":"ContainerStarted","Data":"f3a06798b232a524dbf3af6d04b4ca475b0bb3fea7eefb61baaf283d1274e092"}
Nov 03 22:44:51.628599 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:51.628557    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" event={"ID":"2b3b44d3-d34b-4837-bd03-ade32eb4a473","Type":"ContainerStarted","Data":"e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517"}
Nov 03 22:44:51.805036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:51.804956    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podStartSLOduration=2.804938186 podStartE2EDuration="2.804938186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:44:51.753757768 +0000 UTC m=+1315.194295029" watchObservedRunningTime="2026-11-03 22:44:51.804938186 +0000 UTC m=+1315.245475449"
Nov 03 22:44:52.098374 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:52.098335    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:44:52.099718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:52.099697    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:44:52.099817 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:52.099742    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:44:53.098854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:53.098809    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:44:53.099345 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:53.098873    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:44:54.098271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:54.098237    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:44:54.098573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:54.098540    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:44:55.099291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:55.099253    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:44:55.099873 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:55.099841    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:44:56.027028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.026971    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:44:56.099619 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.099574    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:44:56.101779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.101732    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:44:56.110196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.110147    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:56.220855 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.220801    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:56.221449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.221418    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:44:56.273833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.273777    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:44:56.274040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.273857    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:56.293775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.291176    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:56.462885 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.462829    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:44:56.463980 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.463140    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:56.515820 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.515775    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:56.968755 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.928000    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:57.088703 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:56.994155    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:57.176743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.151560    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:57.268022 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.177533    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:57.268022 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.177871    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:57.268022 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.178969    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/health\": context deadline exceeded"
Nov 03 22:44:57.268022 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.198190    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:44:57.268022 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.198242    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:44:57.304707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.299809    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:44:57.304707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.300207    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:44:57.304707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.300479    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Nov 03 22:44:57.645425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.466801    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:57.645425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.467306    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:44:57.645425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.635227    5647 patch_prober.go:29] interesting pod/thanos-querier-b46bc6cb-p4dg9 container/kube-rbac-proxy-web namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.12:9091/-/ready\": context deadline exceeded" start-of-body=
Nov 03 22:44:57.645425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.635282    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-monitoring/thanos-querier-b46bc6cb-p4dg9" podUID="3ead9dc1-5785-432c-a28d-6dd002c4dc27" containerName="kube-rbac-proxy-web" probeResult="failure" output="Get \"https://10.134.0.12:9091/-/ready\": context deadline exceeded"
Nov 03 22:44:57.724763 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.714591    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Nov 03 22:44:57.819146 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.811932    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"]
Nov 03 22:44:57.885697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.878230    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:44:57.885697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.878288    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:44:57.885697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.878309    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"]
Nov 03 22:44:57.885697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:57.883197    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:44:58.141696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.110463    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-468d8d1b-dockercfg-zd49h\""
Nov 03 22:44:58.152832 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.152783    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.116938    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-468d8d1b\""
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.186010    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.186066    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.186085    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"]
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.186131    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.186164    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.186183    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"]
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.186229    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.186607    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.187572    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.190342    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.194797    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.163887    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:58.198848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.195474    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:44:58.242772 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.231325    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:44:58.252706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.247794    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:44:58.350693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.344376    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:44:58.350693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.344610    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:58.350693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.344821    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Nov 03 22:44:58.350693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.345163    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:44:58.490722 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.483809    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:44:58.490722 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.483883    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:58.502634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.502543    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:44:58.503693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.503646    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:44:58.601775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.585121    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldwmc\" (UniqueName: \"kubernetes.io/projected/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-kube-api-access-ldwmc\") pod \"payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x\" (UID: \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:44:58.601775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.588782    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x\" (UID: \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:44:58.717697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.716291    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23f52be-ec70-459e-b6a0-3c599e68584f-plugins-config-volume\") pod \"payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz\" (UID: \"c23f52be-ec70-459e-b6a0-3c599e68584f\") " pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:44:58.717697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.716389    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltqm9\" (UniqueName: \"kubernetes.io/projected/c23f52be-ec70-459e-b6a0-3c599e68584f-kube-api-access-ltqm9\") pod \"payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz\" (UID: \"c23f52be-ec70-459e-b6a0-3c599e68584f\") " pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:44:58.717697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.716512    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwmc\" (UniqueName: \"kubernetes.io/projected/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-kube-api-access-ldwmc\") pod \"payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x\" (UID: \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:44:58.717697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.716634    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x\" (UID: \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:44:58.727695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.719572    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x\" (UID: \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:44:58.757796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.720775    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:58.843694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.840511    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:44:58.941691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.941123    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:58.941691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.888998    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23f52be-ec70-459e-b6a0-3c599e68584f-plugins-config-volume\") pod \"payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz\" (UID: \"c23f52be-ec70-459e-b6a0-3c599e68584f\") " pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:44:58.941691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.900433    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldwmc\" (UniqueName: \"kubernetes.io/projected/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-kube-api-access-ldwmc\") pod \"payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x\" (UID: \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:44:58.941691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.888053    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23f52be-ec70-459e-b6a0-3c599e68584f-plugins-config-volume\") pod \"payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz\" (UID: \"c23f52be-ec70-459e-b6a0-3c599e68584f\") " pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:44:58.942063 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:58.941698    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltqm9\" (UniqueName: \"kubernetes.io/projected/c23f52be-ec70-459e-b6a0-3c599e68584f-kube-api-access-ltqm9\") pod \"payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz\" (UID: \"c23f52be-ec70-459e-b6a0-3c599e68584f\") " pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:44:59.215474 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.215423    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:44:59.233570 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.233528    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:44:59.234492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.234459    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:44:59.235273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.235247    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:44:59.238742 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.238715    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltqm9\" (UniqueName: \"kubernetes.io/projected/c23f52be-ec70-459e-b6a0-3c599e68584f-kube-api-access-ltqm9\") pod \"payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz\" (UID: \"c23f52be-ec70-459e-b6a0-3c599e68584f\") " pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:44:59.245778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.245037    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:59.246190 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.245422    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:44:59.253789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.253745    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:59.362455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.362418    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:44:59.458715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.442426    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:44:59.481752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.477335    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:44:59.481752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.477408    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:44:59.548541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.548501    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:59.549178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.549149    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:44:59.782779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.772179    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:44:59.782779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.772253    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:44:59.906948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.879041    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:44:59.906948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.887728    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:44:59.937697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.913135    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Nov 03 22:44:59.972251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:44:59.972200    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:00.170955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.170906    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:45:00.171255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.171217    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:00.209838 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.209799    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:45:00.210159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.210113    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:45:00.307243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.290425    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:00.307959 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.307911    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:45:00.309032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.308989    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:45:00.402897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.312795    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:45:00.582701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.566808    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:00.582701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.567219    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:45:00.887831 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.887775    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:00.888229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.888193    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:01.028734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.014486    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/health\": context deadline exceeded"
Nov 03 22:45:01.028734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.976491    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Nov 03 22:45:01.028734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:00.976201    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/health\": context deadline exceeded"
Nov 03 22:45:01.028734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.015624    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:01.079832 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.079796    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:45:01.083828 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.083783    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:01.124711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.123930    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:45:01.124711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.123988    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:45:01.298697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.290912    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:01.318897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.318846    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:45:01.374901 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.374844    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:01.509215 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.507210    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:01.581770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.580829    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:45:01.616708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.593512    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"]
Nov 03 22:45:01.616708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.616168    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:01.670100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.669965    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:45:01.713696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.709019    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:01.723696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.716368    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8567a138-4e13-477c-97b6-dc495e6d2e39-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d\" (UID: \"8567a138-4e13-477c-97b6-dc495e6d2e39\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:01.727696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.724153    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66pnc\" (UniqueName: \"kubernetes.io/projected/8567a138-4e13-477c-97b6-dc495e6d2e39-kube-api-access-66pnc\") pod \"maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d\" (UID: \"8567a138-4e13-477c-97b6-dc495e6d2e39\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:01.780789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.780739    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"]
Nov 03 22:45:01.912545 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.798841    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:01.912545 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.841069    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66pnc\" (UniqueName: \"kubernetes.io/projected/8567a138-4e13-477c-97b6-dc495e6d2e39-kube-api-access-66pnc\") pod \"maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d\" (UID: \"8567a138-4e13-477c-97b6-dc495e6d2e39\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:01.912545 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.841324    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8567a138-4e13-477c-97b6-dc495e6d2e39-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d\" (UID: \"8567a138-4e13-477c-97b6-dc495e6d2e39\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:01.912545 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:01.842447    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8567a138-4e13-477c-97b6-dc495e6d2e39-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d\" (UID: \"8567a138-4e13-477c-97b6-dc495e6d2e39\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:02.285742 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.274890    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p2s2g\""
Nov 03 22:45:02.299691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.291405    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:45:02.299691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.291500    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:45:02.338691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.338455    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66pnc\" (UniqueName: \"kubernetes.io/projected/8567a138-4e13-477c-97b6-dc495e6d2e39-kube-api-access-66pnc\") pod \"maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d\" (UID: \"8567a138-4e13-477c-97b6-dc495e6d2e39\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:02.432713 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.430163    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:45:02.462712 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.456221    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:02.555373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.527153    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:02.555373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.537936    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:02.555373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.540028    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Nov 03 22:45:02.592648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.592602    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:02.626700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.600051    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:02.626700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.605069    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:02.681273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.649052    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:02.681273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.675762    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:45:02.756891 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.753369    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:02.789551 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.789511    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzggc\" (UniqueName: \"kubernetes.io/projected/2b49326a-0663-4a3c-a872-a0278d042fc9-kube-api-access-qzggc\") pod \"maas-api-key-cleanup-29895765-c5bv7\" (UID: \"2b49326a-0663-4a3c-a872-a0278d042fc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:02.790053 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:02.790026    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b49326a-0663-4a3c-a872-a0278d042fc9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895765-c5bv7\" (UID: \"2b49326a-0663-4a3c-a872-a0278d042fc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:03.090765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.085152    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b49326a-0663-4a3c-a872-a0278d042fc9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895765-c5bv7\" (UID: \"2b49326a-0663-4a3c-a872-a0278d042fc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:03.090765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.085377    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzggc\" (UniqueName: \"kubernetes.io/projected/2b49326a-0663-4a3c-a872-a0278d042fc9-kube-api-access-qzggc\") pod \"maas-api-key-cleanup-29895765-c5bv7\" (UID: \"2b49326a-0663-4a3c-a872-a0278d042fc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:03.102793 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.091334    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:45:03.102793 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.091399    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:45:03.131498 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.131447    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b49326a-0663-4a3c-a872-a0278d042fc9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895765-c5bv7\" (UID: \"2b49326a-0663-4a3c-a872-a0278d042fc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:03.155227 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.155167    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"]
Nov 03 22:45:03.167262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.167230    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"]
Nov 03 22:45:03.290571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.282506    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzggc\" (UniqueName: \"kubernetes.io/projected/2b49326a-0663-4a3c-a872-a0278d042fc9-kube-api-access-qzggc\") pod \"maas-api-key-cleanup-29895765-c5bv7\" (UID: \"2b49326a-0663-4a3c-a872-a0278d042fc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:03.290571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.282835    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:45:03.290571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.282884    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:45:03.305796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.305752    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:03.430249 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.430166    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:45:03.431020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.430978    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:03.506947 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.506883    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:45:03.621737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.610150    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:03.671694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.670896    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:45:03.746861 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.746794    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:03.790803 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.766429    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:03.888865 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.888826    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:03.900019 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:03.899983    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:45:04.052305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.052257    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:45:04.056134 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.054758    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:04.066802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.066778    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:45:04.068583 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.068563    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:45:04.098927 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.085615    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"]
Nov 03 22:45:04.098927 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.087604    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:45:04.128075 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.128030    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 03 22:45:04.128853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.128820    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 03 22:45:04.140269 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.140247    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:45:04.157065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.156570    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:45:04.243114 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.228585    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x" event={"ID":"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3","Type":"ContainerStarted","Data":"e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42"}
Nov 03 22:45:04.243114 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.228645    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x" event={"ID":"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3","Type":"ContainerStarted","Data":"6b66f2e54dc967521f678cd16a6c513c3a356336377921fbceb103c2b0fcf09b"}
Nov 03 22:45:04.243114 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.229550    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:45:04.243114 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.236367    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz" event={"ID":"c23f52be-ec70-459e-b6a0-3c599e68584f","Type":"ContainerStarted","Data":"284fe5bddf63fb1e8b5873d011485c916ceed76a411ff036d7774a4191be4f49"}
Nov 03 22:45:04.262016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.251715    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"]
Nov 03 22:45:04.559310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.559282    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"]
Nov 03 22:45:04.562128 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:45:04.562097    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8567a138_4e13_477c_97b6_dc495e6d2e39.slice/crio-013cb1d8d681e936125876bd14bc49d403d7e2217e7fa816f1fd50fb40ff7f79 WatchSource:0}: Error finding container 013cb1d8d681e936125876bd14bc49d403d7e2217e7fa816f1fd50fb40ff7f79: Status 404 returned error can't find the container with id 013cb1d8d681e936125876bd14bc49d403d7e2217e7fa816f1fd50fb40ff7f79
Nov 03 22:45:04.564903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.564887    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:45:04.603070 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:04.602980    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"]
Nov 03 22:45:04.604910 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:45:04.604063    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b49326a_0663_4a3c_a872_a0278d042fc9.slice/crio-23e721b394e3f66ab43c62f3b295eecb6f7c85b6a94f203914f0abc9b13cefd8 WatchSource:0}: Error finding container 23e721b394e3f66ab43c62f3b295eecb6f7c85b6a94f203914f0abc9b13cefd8: Status 404 returned error can't find the container with id 23e721b394e3f66ab43c62f3b295eecb6f7c85b6a94f203914f0abc9b13cefd8
Nov 03 22:45:05.045125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:05.045062    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x" podStartSLOduration=8.045041649 podStartE2EDuration="8.045041649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:05.037192688 +0000 UTC m=+1328.477729949" watchObservedRunningTime="2026-11-03 22:45:05.045041649 +0000 UTC m=+1328.485578909"
Nov 03 22:45:05.098278 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:05.098239    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:45:05.252251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:05.252216    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" event={"ID":"2b49326a-0663-4a3c-a872-a0278d042fc9","Type":"ContainerStarted","Data":"23e721b394e3f66ab43c62f3b295eecb6f7c85b6a94f203914f0abc9b13cefd8"}
Nov 03 22:45:05.253705 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:05.253654    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" event={"ID":"8567a138-4e13-477c-97b6-dc495e6d2e39","Type":"ContainerStarted","Data":"013cb1d8d681e936125876bd14bc49d403d7e2217e7fa816f1fd50fb40ff7f79"}
Nov 03 22:45:05.255640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:05.255576    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz" event={"ID":"c23f52be-ec70-459e-b6a0-3c599e68584f","Type":"ContainerStarted","Data":"5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7"}
Nov 03 22:45:05.255810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:05.255705    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:45:06.099195 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:06.099142    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:45:06.099765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:06.099215    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:07.100241 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.100203    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:45:07.100647 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.100277    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:07.101407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.101388    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:45:07.102294 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.102265    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:45:07.223108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.223058    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz" podStartSLOduration=10.223044619 podStartE2EDuration="10.223044619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:05.513558132 +0000 UTC m=+1328.954095392" watchObservedRunningTime="2026-11-03 22:45:07.223044619 +0000 UTC m=+1330.663581878"
Nov 03 22:45:07.795567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.795532    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"]
Nov 03 22:45:07.800519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.800490    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:07.853523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.853481    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 22:45:07.857741 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.857712    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"]
Nov 03 22:45:07.926846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.926802    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:07.927031 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.926863    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8crvd\" (UniqueName: \"kubernetes.io/projected/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-kube-api-access-8crvd\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:07.927143 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:07.927122    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:08.030728 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:08.028476    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:08.030728 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:08.028547    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8crvd\" (UniqueName: \"kubernetes.io/projected/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-kube-api-access-8crvd\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:08.030728 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:08.028910    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:08.038726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:08.035476    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:08.061273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:08.059120    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:08.121489 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:08.121416    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8crvd\" (UniqueName: \"kubernetes.io/projected/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-kube-api-access-8crvd\") pod \"maas-api-revoke-sub-3c277faf-9mtv9\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:08.418412 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:08.418340    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:08.805720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:08.805684    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"]
Nov 03 22:45:08.807046 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:45:08.807016    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9bb1d9_22e7_4de7_876f_e72afb5f485a.slice/crio-091b1f50cfd192443536f8578366e1a85438284e68ccae35f766e2807bf75266 WatchSource:0}: Error finding container 091b1f50cfd192443536f8578366e1a85438284e68ccae35f766e2807bf75266: Status 404 returned error can't find the container with id 091b1f50cfd192443536f8578366e1a85438284e68ccae35f766e2807bf75266
Nov 03 22:45:09.309263 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:09.309228    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9" event={"ID":"fa9bb1d9-22e7-4de7-876f-e72afb5f485a","Type":"ContainerStarted","Data":"091b1f50cfd192443536f8578366e1a85438284e68ccae35f766e2807bf75266"}
Nov 03 22:45:10.317414 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:10.317374    5647 generic.go:356] "Generic (PLEG): container finished" podID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerID="d5da40ac0b78e21ff3b89b45e3138a110d0c2b8cbe199f141d16d30dcf77bd2f" exitCode=60
Nov 03 22:45:10.317858 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:10.317461    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" event={"ID":"8567a138-4e13-477c-97b6-dc495e6d2e39","Type":"ContainerDied","Data":"d5da40ac0b78e21ff3b89b45e3138a110d0c2b8cbe199f141d16d30dcf77bd2f"}
Nov 03 22:45:10.317858 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:10.317781    5647 scope.go:122] "RemoveContainer" containerID="d5da40ac0b78e21ff3b89b45e3138a110d0c2b8cbe199f141d16d30dcf77bd2f"
Nov 03 22:45:11.325343 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:11.325312    5647 generic.go:356] "Generic (PLEG): container finished" podID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerID="fecfb1593311d55cb8649be8ec678b0572e94029b3a33c422ac63a2b34de4fa8" exitCode=60
Nov 03 22:45:11.325781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:11.325400    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" event={"ID":"2b49326a-0663-4a3c-a872-a0278d042fc9","Type":"ContainerDied","Data":"fecfb1593311d55cb8649be8ec678b0572e94029b3a33c422ac63a2b34de4fa8"}
Nov 03 22:45:11.325781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:11.325720    5647 scope.go:122] "RemoveContainer" containerID="fecfb1593311d55cb8649be8ec678b0572e94029b3a33c422ac63a2b34de4fa8"
Nov 03 22:45:11.327391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:11.327369    5647 generic.go:356] "Generic (PLEG): container finished" podID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerID="820d96c0e6cbb19750cd97299efe39e6c8af3db3add2887d0a01c3656be86aa4" exitCode=60
Nov 03 22:45:11.327493 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:11.327478    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" event={"ID":"8567a138-4e13-477c-97b6-dc495e6d2e39","Type":"ContainerDied","Data":"820d96c0e6cbb19750cd97299efe39e6c8af3db3add2887d0a01c3656be86aa4"}
Nov 03 22:45:11.327544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:11.327523    5647 scope.go:122] "RemoveContainer" containerID="d5da40ac0b78e21ff3b89b45e3138a110d0c2b8cbe199f141d16d30dcf77bd2f"
Nov 03 22:45:11.327825 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:11.327806    5647 scope.go:122] "RemoveContainer" containerID="820d96c0e6cbb19750cd97299efe39e6c8af3db3add2887d0a01c3656be86aa4"
Nov 03 22:45:11.328178 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:45:11.328128    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d_odh-ai-gateway-infra(8567a138-4e13-477c-97b6-dc495e6d2e39)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39"
Nov 03 22:45:12.337018 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:12.336991    5647 scope.go:122] "RemoveContainer" containerID="820d96c0e6cbb19750cd97299efe39e6c8af3db3add2887d0a01c3656be86aa4"
Nov 03 22:45:12.337408 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:45:12.337350    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d_odh-ai-gateway-infra(8567a138-4e13-477c-97b6-dc495e6d2e39)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39"
Nov 03 22:45:12.338825 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:12.338801    5647 generic.go:356] "Generic (PLEG): container finished" podID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerID="5eb69c673f2c04b5d2a8694efe20e99c6eb3d56eeba7b928c18db5f74eabb44c" exitCode=60
Nov 03 22:45:12.338958 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:12.338844    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" event={"ID":"2b49326a-0663-4a3c-a872-a0278d042fc9","Type":"ContainerDied","Data":"5eb69c673f2c04b5d2a8694efe20e99c6eb3d56eeba7b928c18db5f74eabb44c"}
Nov 03 22:45:12.338958 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:12.338884    5647 scope.go:122] "RemoveContainer" containerID="fecfb1593311d55cb8649be8ec678b0572e94029b3a33c422ac63a2b34de4fa8"
Nov 03 22:45:12.339178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:12.339160    5647 scope.go:122] "RemoveContainer" containerID="5eb69c673f2c04b5d2a8694efe20e99c6eb3d56eeba7b928c18db5f74eabb44c"
Nov 03 22:45:12.339482 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:45:12.339458    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895765-c5bv7_odh-ai-gateway-infra(2b49326a-0663-4a3c-a872-a0278d042fc9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9"
Nov 03 22:45:13.347871 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:13.347834    5647 generic.go:356] "Generic (PLEG): container finished" podID="fa9bb1d9-22e7-4de7-876f-e72afb5f485a" containerID="3ebf3f8516c733ce735248f34d1f4e5c03917449794a2edfd582e43cf12f7063" exitCode=0
Nov 03 22:45:13.348346 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:13.347934    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9" event={"ID":"fa9bb1d9-22e7-4de7-876f-e72afb5f485a","Type":"ContainerDied","Data":"3ebf3f8516c733ce735248f34d1f4e5c03917449794a2edfd582e43cf12f7063"}
Nov 03 22:45:13.350289 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:13.350269    5647 scope.go:122] "RemoveContainer" containerID="5eb69c673f2c04b5d2a8694efe20e99c6eb3d56eeba7b928c18db5f74eabb44c"
Nov 03 22:45:13.350601 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:45:13.350582    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895765-c5bv7_odh-ai-gateway-infra(2b49326a-0663-4a3c-a872-a0278d042fc9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9"
Nov 03 22:45:14.522189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.522163    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:14.637946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.637862    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-xks-ca\") pod \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") "
Nov 03 22:45:14.638122 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.638030    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-openshift-ca\") pod \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") "
Nov 03 22:45:14.638122 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.638117    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-kube-api-access-8crvd\" (UniqueName: \"kubernetes.io/projected/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-kube-api-access-8crvd\") pod \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\" (UID: \"fa9bb1d9-22e7-4de7-876f-e72afb5f485a\") "
Nov 03 22:45:14.638490 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.638382    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-openshift-ca" pod "fa9bb1d9-22e7-4de7-876f-e72afb5f485a" (UID: "fa9bb1d9-22e7-4de7-876f-e72afb5f485a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:45:14.638882 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.638860    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-openshift-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:45:14.641115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.641089    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-kube-api-access-8crvd" pod "fa9bb1d9-22e7-4de7-876f-e72afb5f485a" (UID: "fa9bb1d9-22e7-4de7-876f-e72afb5f485a"). InnerVolumeSpecName "kube-api-access-8crvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:45:14.642208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.642187    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-xks-ca" pod "fa9bb1d9-22e7-4de7-876f-e72afb5f485a" (UID: "fa9bb1d9-22e7-4de7-876f-e72afb5f485a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:45:14.740679 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.740647    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-maas-api-xks-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:45:14.740830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:14.740693    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8crvd\" (UniqueName: \"kubernetes.io/projected/fa9bb1d9-22e7-4de7-876f-e72afb5f485a-kube-api-access-8crvd\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:45:15.366872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:15.366841    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"
Nov 03 22:45:15.367047 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:15.366905    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9" event={"ID":"fa9bb1d9-22e7-4de7-876f-e72afb5f485a","Type":"ContainerDied","Data":"091b1f50cfd192443536f8578366e1a85438284e68ccae35f766e2807bf75266"}
Nov 03 22:45:15.367047 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:15.366973    5647 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="091b1f50cfd192443536f8578366e1a85438284e68ccae35f766e2807bf75266"
Nov 03 22:45:16.045730 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:45:16.045689    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:45:16.267613 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:16.267583    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:45:16.267849 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:16.267702    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:45:20.752861 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.752822    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"]
Nov 03 22:45:20.753579 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.753562    5647 cpu_manager.go:414] "containerMap: removing container" podUID="fa9bb1d9-22e7-4de7-876f-e72afb5f485a" containerName="revoke-keys"
Nov 03 22:45:20.753624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.753582    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa9bb1d9-22e7-4de7-876f-e72afb5f485a" containerName="revoke-keys"
Nov 03 22:45:20.753757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.753747    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa9bb1d9-22e7-4de7-876f-e72afb5f485a" containerName="revoke-keys"
Nov 03 22:45:20.757649 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.757631    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.802229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.802201    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"]
Nov 03 22:45:20.806719 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.806698    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.838913 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.838875    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839082 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.838943    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.839159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839108    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839150    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.839269 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839190    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839269 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839223    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.839269 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839251    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839269 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839267    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839319    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839334    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtvfg\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-kube-api-access-gtvfg\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.839469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839359    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.839469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839408    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839427    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.839756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839489    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.839756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839519    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cg82\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-kube-api-access-6cg82\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839546    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.839756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839560    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.839756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.839617    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0e0cb9a-b472-4d06-89f6-725712c83b89-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.860136 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.860103    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"]
Nov 03 22:45:20.860278 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.860187    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"]
Nov 03 22:45:20.941633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.941600    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.941860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.941658    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.941860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.941721    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.941860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.941779    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.941860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.941799    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cg82\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-kube-api-access-6cg82\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.941860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.941818    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.941860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.941851    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.941915    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0e0cb9a-b472-4d06-89f6-725712c83b89-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942004    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942046    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942098    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942101    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942101    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942155    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942160    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942235    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942277    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942324    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942350    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942424    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942447    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-data\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942576 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942461    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942761    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942768    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-certs\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.942948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942457    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtvfg\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-kube-api-access-gtvfg\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.942948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942809    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0e0cb9a-b472-4d06-89f6-725712c83b89-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.943134 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.942951    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-credential-socket\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.943134 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.943115    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istiod-ca-cert\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.944621 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.944600    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.945068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.945045    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-envoy\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:20.945068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.945051    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:20.945382 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:20.945358    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-podinfo\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:21.003936 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.003852    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:21.004333 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.004313    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-token\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:21.004503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.004474    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtvfg\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-kube-api-access-gtvfg\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:21.005640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.005619    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cg82\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-kube-api-access-6cg82\") pod \"e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") " pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:21.071830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.071799    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:21.123770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.123738    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:21.197380 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.197318    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"]
Nov 03 22:45:21.210434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.210405    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.248609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.247984    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.248609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.248052    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.248609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.248237    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.248609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.248327    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.248609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.248385    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.248609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.248431    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jshnz\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-kube-api-access-jshnz\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.248609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.248491    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.248609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.248554    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/446dfad7-dee2-44a7-83f7-9cee265d5d24-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.249197 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.248643    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.253207 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.253180    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"]
Nov 03 22:45:21.350422 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.350344    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.350422 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.350383    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.350656 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.350489    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.350656 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.350559    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.350656 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.350605    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.350960 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.350933    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.351073 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.350983    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.351073 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.350656    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jshnz\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-kube-api-access-jshnz\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.351181 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.351085    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.351233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.351188    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/446dfad7-dee2-44a7-83f7-9cee265d5d24-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.351357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.351341    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.351421 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.351400    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.351768 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.351747    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.352055 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.352030    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/446dfad7-dee2-44a7-83f7-9cee265d5d24-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.353059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.353025    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.353364 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.353338    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.411337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.411307    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.411503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.411407    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jshnz\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-kube-api-access-jshnz\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.503442 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.503417    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"]
Nov 03 22:45:21.504336 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:45:21.504314    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e0cb9a_b472_4d06_89f6_725712c83b89.slice/crio-d7cab3a483aaa102b98086f7680553675b2a22edfdd24398a6670efee85aa7c7 WatchSource:0}: Error finding container d7cab3a483aaa102b98086f7680553675b2a22edfdd24398a6670efee85aa7c7: Status 404 returned error can't find the container with id d7cab3a483aaa102b98086f7680553675b2a22edfdd24398a6670efee85aa7c7
Nov 03 22:45:21.507582 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.507532    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:21.507652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.507616    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:21.507717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.507657    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:21.534805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.534778    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:21.559864 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.559837    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"]
Nov 03 22:45:21.569023 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:45:21.568938    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b06c2b_0aad_497c_a4e9_9d4c5751bdfa.slice/crio-a7675714b5c1d5d474b5c7ef058176a17e507466cb1904933544ce86d12c52ec WatchSource:0}: Error finding container a7675714b5c1d5d474b5c7ef058176a17e507466cb1904933544ce86d12c52ec: Status 404 returned error can't find the container with id a7675714b5c1d5d474b5c7ef058176a17e507466cb1904933544ce86d12c52ec
Nov 03 22:45:21.583487 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.571796    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:21.583487 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.571876    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:21.583487 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.571919    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:21.934562 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:45:21.934533    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod446dfad7_dee2_44a7_83f7_9cee265d5d24.slice/crio-c63aeb6feb77c3f2aa92f4710c775f9b25f49ecfd36b6fc60f694a7ecd3bb1d7 WatchSource:0}: Error finding container c63aeb6feb77c3f2aa92f4710c775f9b25f49ecfd36b6fc60f694a7ecd3bb1d7: Status 404 returned error can't find the container with id c63aeb6feb77c3f2aa92f4710c775f9b25f49ecfd36b6fc60f694a7ecd3bb1d7
Nov 03 22:45:21.935250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.935221    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"]
Nov 03 22:45:21.937308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.937276    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:21.937413 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.937381    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:21.937477 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:21.937443    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:22.441694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.441387    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" event={"ID":"446dfad7-dee2-44a7-83f7-9cee265d5d24","Type":"ContainerStarted","Data":"06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3"}
Nov 03 22:45:22.441694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.441452    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" event={"ID":"446dfad7-dee2-44a7-83f7-9cee265d5d24","Type":"ContainerStarted","Data":"c63aeb6feb77c3f2aa92f4710c775f9b25f49ecfd36b6fc60f694a7ecd3bb1d7"}
Nov 03 22:45:22.445691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.443283    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" event={"ID":"a0e0cb9a-b472-4d06-89f6-725712c83b89","Type":"ContainerStarted","Data":"d24d865d32e3eec1bd899f669185b8998c3198c5bb279f55302685c1bb121eab"}
Nov 03 22:45:22.445691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.443331    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" event={"ID":"a0e0cb9a-b472-4d06-89f6-725712c83b89","Type":"ContainerStarted","Data":"d7cab3a483aaa102b98086f7680553675b2a22edfdd24398a6670efee85aa7c7"}
Nov 03 22:45:22.445691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.445065    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" event={"ID":"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa","Type":"ContainerStarted","Data":"e2431110e7b1da9defe3bdc6a3858f2f8b77f66667da8f835aeb41ffaf02b2c8"}
Nov 03 22:45:22.445691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.445107    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" event={"ID":"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa","Type":"ContainerStarted","Data":"a7675714b5c1d5d474b5c7ef058176a17e507466cb1904933544ce86d12c52ec"}
Nov 03 22:45:22.536749 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.535021    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:22.566218 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.566164    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podStartSLOduration=1.5661511259999998 podStartE2EDuration="1.566151126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:22.563288084 +0000 UTC m=+1346.003825343" watchObservedRunningTime="2026-11-03 22:45:22.566151126 +0000 UTC m=+1346.006688458"
Nov 03 22:45:22.700323 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.700233    5647 scope.go:122] "RemoveContainer" containerID="820d96c0e6cbb19750cd97299efe39e6c8af3db3add2887d0a01c3656be86aa4"
Nov 03 22:45:22.720314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.720288    5647 scope.go:122] "RemoveContainer" containerID="5eb69c673f2c04b5d2a8694efe20e99c6eb3d56eeba7b928c18db5f74eabb44c"
Nov 03 22:45:22.733545 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.731407    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podStartSLOduration=2.731385866 podStartE2EDuration="2.731385866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:22.727458571 +0000 UTC m=+1346.167995842" watchObservedRunningTime="2026-11-03 22:45:22.731385866 +0000 UTC m=+1346.171923124"
Nov 03 22:45:22.856772 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:22.854760    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podStartSLOduration=2.854738452 podStartE2EDuration="2.854738452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:22.850093422 +0000 UTC m=+1346.290630682" watchObservedRunningTime="2026-11-03 22:45:22.854738452 +0000 UTC m=+1346.295275708"
Nov 03 22:45:23.076704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.074061    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:23.092706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.087808    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:45:23.092706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.087872    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:45:23.128868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.127125    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:23.128868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.128747    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 03 22:45:23.128868 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.128802    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 03 22:45:23.462045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.462008    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" event={"ID":"8567a138-4e13-477c-97b6-dc495e6d2e39","Type":"ContainerStarted","Data":"4000f934f16d9dd4cf22075975b7543d25542759fb52fa07da2a8153fd332fbf"}
Nov 03 22:45:23.465444 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.465402    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" event={"ID":"2b49326a-0663-4a3c-a872-a0278d042fc9","Type":"ContainerStarted","Data":"422c7bd848890858af662624a394ec7869fbdf40bf70dc720ad849dc8b712c4d"}
Nov 03 22:45:23.551746 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.551708    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:45:23.551957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.551757    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:45:23.605754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:23.591699    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" podStartSLOduration=18.970999565 podStartE2EDuration="23.591659715s" totalImagesPullingTime="4.62066015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:23.588132482 +0000 UTC m=+1347.028669743" watchObservedRunningTime="2026-11-03 22:45:23.591659715 +0000 UTC m=+1347.032196974"
Nov 03 22:45:24.086696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.083054    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:45:24.086696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.083115    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:45:24.134690 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.132156    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 03 22:45:24.134690 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.132215    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 03 22:45:24.535695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.535323    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:45:24.535695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.535379    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:45:24.547694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.537527    5647 generic.go:356] "Generic (PLEG): container finished" podID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerID="422c7bd848890858af662624a394ec7869fbdf40bf70dc720ad849dc8b712c4d" exitCode=60
Nov 03 22:45:24.547694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.537643    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" event={"ID":"2b49326a-0663-4a3c-a872-a0278d042fc9","Type":"ContainerDied","Data":"422c7bd848890858af662624a394ec7869fbdf40bf70dc720ad849dc8b712c4d"}
Nov 03 22:45:24.549691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.548794    5647 scope.go:122] "RemoveContainer" containerID="5eb69c673f2c04b5d2a8694efe20e99c6eb3d56eeba7b928c18db5f74eabb44c"
Nov 03 22:45:24.549691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.549419    5647 scope.go:122] "RemoveContainer" containerID="422c7bd848890858af662624a394ec7869fbdf40bf70dc720ad849dc8b712c4d"
Nov 03 22:45:24.549909 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:45:24.549881    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895765-c5bv7_odh-ai-gateway-infra(2b49326a-0663-4a3c-a872-a0278d042fc9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9"
Nov 03 22:45:24.552006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.551933    5647 generic.go:356] "Generic (PLEG): container finished" podID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerID="4000f934f16d9dd4cf22075975b7543d25542759fb52fa07da2a8153fd332fbf" exitCode=60
Nov 03 22:45:24.552514 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.552491    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" event={"ID":"8567a138-4e13-477c-97b6-dc495e6d2e39","Type":"ContainerDied","Data":"4000f934f16d9dd4cf22075975b7543d25542759fb52fa07da2a8153fd332fbf"}
Nov 03 22:45:24.553592 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.553574    5647 scope.go:122] "RemoveContainer" containerID="4000f934f16d9dd4cf22075975b7543d25542759fb52fa07da2a8153fd332fbf"
Nov 03 22:45:24.553821 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:45:24.553800    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d_odh-ai-gateway-infra(8567a138-4e13-477c-97b6-dc495e6d2e39)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39"
Nov 03 22:45:24.564285 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.564268    5647 scope.go:122] "RemoveContainer" containerID="820d96c0e6cbb19750cd97299efe39e6c8af3db3add2887d0a01c3656be86aa4"
Nov 03 22:45:24.608701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.607834    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" podStartSLOduration=18.878980582 podStartE2EDuration="24.607814591s" totalImagesPullingTime="5.728834009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:45:23.751020747 +0000 UTC m=+1347.191558006" watchObservedRunningTime="2026-11-03 22:45:24.607814591 +0000 UTC m=+1348.048351850"
Nov 03 22:45:24.651529 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.639620    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"]
Nov 03 22:45:24.760265 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:24.760233    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"]
Nov 03 22:45:25.080197 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:25.080163    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:45:25.080469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:25.080440    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:45:25.139280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:25.139163    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 03 22:45:25.139280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:25.139219    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 03 22:45:25.536138 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:25.536103    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:45:25.536426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:25.536398    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:45:26.126755 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.125640    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:45:26.126755 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.125721    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:45:26.158035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.127555    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 03 22:45:26.158035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.127597    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 03 22:45:26.372702 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.359370    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:26.461694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.453270    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b49326a-0663-4a3c-a872-a0278d042fc9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b49326a-0663-4a3c-a872-a0278d042fc9-maas-api-service-ca\") pod \"2b49326a-0663-4a3c-a872-a0278d042fc9\" (UID: \"2b49326a-0663-4a3c-a872-a0278d042fc9\") "
Nov 03 22:45:26.461694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.453373    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b49326a-0663-4a3c-a872-a0278d042fc9-kube-api-access-qzggc\" (UniqueName: \"kubernetes.io/projected/2b49326a-0663-4a3c-a872-a0278d042fc9-kube-api-access-qzggc\") pod \"2b49326a-0663-4a3c-a872-a0278d042fc9\" (UID: \"2b49326a-0663-4a3c-a872-a0278d042fc9\") "
Nov 03 22:45:26.494601 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.494565    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b49326a-0663-4a3c-a872-a0278d042fc9-maas-api-service-ca" pod "2b49326a-0663-4a3c-a872-a0278d042fc9" (UID: "2b49326a-0663-4a3c-a872-a0278d042fc9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:45:26.500574 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.500552    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:26.537826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.537789    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:45:26.538120 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.538085    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:45:26.573948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.571225    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b49326a-0663-4a3c-a872-a0278d042fc9-maas-api-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:45:26.743771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.691766    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8567a138-4e13-477c-97b6-dc495e6d2e39-kube-api-access-66pnc\" (UniqueName: \"kubernetes.io/projected/8567a138-4e13-477c-97b6-dc495e6d2e39-kube-api-access-66pnc\") pod \"8567a138-4e13-477c-97b6-dc495e6d2e39\" (UID: \"8567a138-4e13-477c-97b6-dc495e6d2e39\") "
Nov 03 22:45:26.743771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.691904    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8567a138-4e13-477c-97b6-dc495e6d2e39-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8567a138-4e13-477c-97b6-dc495e6d2e39-maas-api-service-ca\") pod \"8567a138-4e13-477c-97b6-dc495e6d2e39\" (UID: \"8567a138-4e13-477c-97b6-dc495e6d2e39\") "
Nov 03 22:45:26.743771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.700267    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b49326a-0663-4a3c-a872-a0278d042fc9-kube-api-access-qzggc" pod "2b49326a-0663-4a3c-a872-a0278d042fc9" (UID: "2b49326a-0663-4a3c-a872-a0278d042fc9"). InnerVolumeSpecName "kube-api-access-qzggc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:45:26.745052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.744545    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8567a138-4e13-477c-97b6-dc495e6d2e39-maas-api-service-ca" pod "8567a138-4e13-477c-97b6-dc495e6d2e39" (UID: "8567a138-4e13-477c-97b6-dc495e6d2e39"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:45:26.856702 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.853616    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8567a138-4e13-477c-97b6-dc495e6d2e39-maas-api-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:45:26.856702 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.854043    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzggc\" (UniqueName: \"kubernetes.io/projected/2b49326a-0663-4a3c-a872-a0278d042fc9-kube-api-access-qzggc\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:45:26.867444 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.867410    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"
Nov 03 22:45:26.867728 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.867706    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"
Nov 03 22:45:26.868017 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:26.867985    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8567a138-4e13-477c-97b6-dc495e6d2e39-kube-api-access-66pnc" pod "8567a138-4e13-477c-97b6-dc495e6d2e39" (UID: "8567a138-4e13-477c-97b6-dc495e6d2e39"). InnerVolumeSpecName "kube-api-access-66pnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:45:27.018019 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.008799    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66pnc\" (UniqueName: \"kubernetes.io/projected/8567a138-4e13-477c-97b6-dc495e6d2e39-kube-api-access-66pnc\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:45:27.138071 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.137966    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7" event={"ID":"2b49326a-0663-4a3c-a872-a0278d042fc9","Type":"ContainerDied","Data":"23e721b394e3f66ab43c62f3b295eecb6f7c85b6a94f203914f0abc9b13cefd8"}
Nov 03 22:45:27.152175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.138291    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d" event={"ID":"8567a138-4e13-477c-97b6-dc495e6d2e39","Type":"ContainerDied","Data":"013cb1d8d681e936125876bd14bc49d403d7e2217e7fa816f1fd50fb40ff7f79"}
Nov 03 22:45:27.152175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.138356    5647 scope.go:122] "RemoveContainer" containerID="422c7bd848890858af662624a394ec7869fbdf40bf70dc720ad849dc8b712c4d"
Nov 03 22:45:27.152175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.148013    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:45:27.152175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.148160    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:45:27.164692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.156581    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 03 22:45:27.164692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.156627    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 03 22:45:27.264293 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.264269    5647 scope.go:122] "RemoveContainer" containerID="4000f934f16d9dd4cf22075975b7543d25542759fb52fa07da2a8153fd332fbf"
Nov 03 22:45:27.344770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.342724    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"]
Nov 03 22:45:27.357796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.357233    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895765-c5bv7"]
Nov 03 22:45:27.449012 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.448972    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"]
Nov 03 22:45:27.456455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.455982    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-cbd26c-29895765-mdd8d"]
Nov 03 22:45:27.554921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.554890    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:45:27.555157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:27.555130    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:45:28.078698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:28.075043    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Nov 03 22:45:28.078698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:28.075109    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Nov 03 22:45:28.124644 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:28.124612    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 03 22:45:28.124836 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:28.124663    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 03 22:45:28.535547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:28.535515    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:45:28.536069 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:28.535574    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:45:28.700585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:28.700548    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" path="/var/lib/kubelet/pods/2b49326a-0663-4a3c-a872-a0278d042fc9/volumes"
Nov 03 22:45:28.701145 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:28.701122    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" path="/var/lib/kubelet/pods/8567a138-4e13-477c-97b6-dc495e6d2e39/volumes"
Nov 03 22:45:29.540113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:29.540077    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Nov 03 22:45:29.540792 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:29.540757    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Nov 03 22:45:29.651903 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:29.651868    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:29.653112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:29.653090    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:29.654867 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:29.654846    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 22:45:29.802308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:29.801982    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:29.805046 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:29.803348    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:29.807113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:29.807083    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:45:30.540017 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:30.539981    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:30.540451 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:30.540438    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:30.541287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:30.541267    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:45:35.334113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:35.334079    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:45:37.228996 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:37.228781    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:37.263695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:37.261776    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:37.444105 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:37.444058    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:37.466771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:37.466727    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:37.467357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:37.466989    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:37.514701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:37.513772    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:37.592486 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:37.592443    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.065697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.019264    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Nov 03 22:45:38.231800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.231752    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.257704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.244153    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:45:38.264624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.264575    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.302705 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.275831    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:45:38.461861 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.459909    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.461861 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.461373    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:45:38.480106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.477925    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.480106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.479027    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:45:38.480767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.480421    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:45:38.480767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.480511    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:45:38.514136 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.514096    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.553817 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.553786    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:45:38.605809 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.605759    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.610598 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.610549    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:45:38.665106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.665053    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.834147 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.834109    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:45:38.834470 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.834438    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:38.977921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:38.977807    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.120805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.120763    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": context deadline exceeded"
Nov 03 22:45:39.168244 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.147376    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.169371 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.169344    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:45:39.189165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.154768    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:45:39.190095 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.189491    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.247775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.247638    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.303162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.265006    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:45:39.467950 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.465749    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.524557 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.524457    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.537743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.537700    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.650775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.650731    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.844133 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.669565    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.844571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.826369    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:45:39.847798 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.847774    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:45:39.991706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.990756    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:39.991706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:39.991183    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:45:40.176810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.169409    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Nov 03 22:45:40.311603 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.304334    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:40.311603 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.304641    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:45:40.473424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.473377    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:40.546362 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.538358    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:45:40.546362 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.538844    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:40.673301 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.673268    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:45:40.677429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.677405    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:45:40.847251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.845764    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:40.857492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.857355    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:45:40.933106 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:40.909444    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:45:41.037316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.037270    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/health\": context deadline exceeded"
Nov 03 22:45:41.061695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.047348    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 22:45:41.076963 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.076931    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:45:41.077237 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.077222    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:45:41.077433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.077417    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:45:41.077654 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.077639    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:45:41.230196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.175142    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Nov 03 22:45:41.588048 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.252905    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:45:41.807987 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:41.807960    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:45:42.065611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:42.053263    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:45:46.051231 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:45:46.051192    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:45:58.316350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.316317    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"]
Nov 03 22:45:58.317194 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317176    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:45:58.317257 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317196    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:45:58.317257 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317215    5647 cpu_manager.go:414] "containerMap: removing container" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:45:58.317257 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317220    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:45:58.317257 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317249    5647 cpu_manager.go:414] "containerMap: removing container" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:45:58.317257 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317254    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:45:58.317434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317264    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:45:58.317434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317268    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:45:58.317434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317369    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:45:58.317434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317380    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:45:58.317434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317388    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:45:58.317434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317395    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:45:58.317434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.317404    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:45:58.322318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.322300    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.375303 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.374759    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1cb89500-openshift-default-dockercfg-jn5jf\""
Nov 03 22:45:58.428181 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.428151    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"]
Nov 03 22:45:58.441752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.441717    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-token\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.441937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.441757    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9daf04a2-44f1-4659-b8ce-ad117f28527d-istiod-ca-cert\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.442022 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.441994    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-credential-socket\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.442131 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.442113    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-data\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.442251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.442233    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-podinfo\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.442319 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.442282    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t55x5\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-kube-api-access-t55x5\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.442391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.442376    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-socket\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.442453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.442428    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-envoy\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.442548 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.442527    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-certs\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.593951    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-credential-socket\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.594076    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-data\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.594166    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-podinfo\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.594210    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t55x5\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-kube-api-access-t55x5\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.594266    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-socket\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.594312    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-envoy\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.594405    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-certs\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.594481    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-token\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.594505    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9daf04a2-44f1-4659-b8ce-ad117f28527d-istiod-ca-cert\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.643748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.595341    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9daf04a2-44f1-4659-b8ce-ad117f28527d-istiod-ca-cert\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.644699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.644640    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-credential-socket\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.665696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.646191    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-data\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.682691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.678285    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-certs\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.682691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.678518    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-socket\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.772398 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.771982    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-envoy\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.777970 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.777886    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t55x5\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-kube-api-access-t55x5\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.783351 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.783311    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-podinfo\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.864571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.864536    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-token\") pod \"e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") " pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:58.935998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:58.935962    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:45:59.380424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:59.380399    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"]
Nov 03 22:45:59.381339 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:45:59.381318    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9daf04a2_44f1_4659_b8ce_ad117f28527d.slice/crio-8d77a0e827c0e50c2521111d7c902d82b44e2f17b4bb06c3ab0fd4fa6e3179e5 WatchSource:0}: Error finding container 8d77a0e827c0e50c2521111d7c902d82b44e2f17b4bb06c3ab0fd4fa6e3179e5: Status 404 returned error can't find the container with id 8d77a0e827c0e50c2521111d7c902d82b44e2f17b4bb06c3ab0fd4fa6e3179e5
Nov 03 22:45:59.383993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:59.383962    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:59.384047 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:59.384029    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:45:59.384081 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:45:59.384056    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:00.257651 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:00.257610    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" event={"ID":"9daf04a2-44f1-4659-b8ce-ad117f28527d","Type":"ContainerStarted","Data":"b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84"}
Nov 03 22:46:00.257862 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:00.257688    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" event={"ID":"9daf04a2-44f1-4659-b8ce-ad117f28527d","Type":"ContainerStarted","Data":"8d77a0e827c0e50c2521111d7c902d82b44e2f17b4bb06c3ab0fd4fa6e3179e5"}
Nov 03 22:46:00.385819 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:00.385746    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podStartSLOduration=2.385730416 podStartE2EDuration="2.385730416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:00.381772968 +0000 UTC m=+1383.822310230" watchObservedRunningTime="2026-11-03 22:46:00.385730416 +0000 UTC m=+1383.826267675"
Nov 03 22:46:00.937469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:00.937437    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:46:00.943426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:00.943401    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:46:01.264856 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:01.264822    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:46:01.265988 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:01.265969    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:46:06.577059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.577022    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"]
Nov 03 22:46:06.577992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.577968    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:46:06.577992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.577990    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8567a138-4e13-477c-97b6-dc495e6d2e39" containerName="cleanup"
Nov 03 22:46:06.578169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.578056    5647 cpu_manager.go:414] "containerMap: removing container" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:46:06.578169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.578064    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:46:06.578284 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.578269    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b49326a-0663-4a3c-a872-a0278d042fc9" containerName="cleanup"
Nov 03 22:46:06.582031 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.582011    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.697216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.697186    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"]
Nov 03 22:46:06.743151 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743117    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05719b59-3d32-4f91-b8f1-fa91e1901161-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.743329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743166    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.743329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743182    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sq54\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-kube-api-access-6sq54\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.743329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743302    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.743461 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743425    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.743526 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743511    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.743594 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743570    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.743652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743613    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.743733 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.743710    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.845589 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.845475    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.845813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.845605    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.845813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.845753    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.845813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.845790    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.845818    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.845885    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.845964    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846161 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.846055    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05719b59-3d32-4f91-b8f1-fa91e1901161-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846161 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.846105    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846161 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.846130    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sq54\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-kube-api-access-6sq54\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.846218    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-data\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.846300    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-certs\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.846699    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-credential-socket\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.846841 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.846822    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05719b59-3d32-4f91-b8f1-fa91e1901161-istiod-ca-cert\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.848414 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.848390    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-envoy\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.848913 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.848892    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-podinfo\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.906906 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.906872    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-token\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:06.907031 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:06.906938    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sq54\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-kube-api-access-6sq54\") pod \"e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") " pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:07.197540 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:07.197442    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:07.634275 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:07.634243    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"]
Nov 03 22:46:07.634785 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:46:07.634748    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05719b59_3d32_4f91_b8f1_fa91e1901161.slice/crio-f70a9b60a6e1a901cea28421aed7a6c4810a793aab210c515fc16ccf6dbd15b6 WatchSource:0}: Error finding container f70a9b60a6e1a901cea28421aed7a6c4810a793aab210c515fc16ccf6dbd15b6: Status 404 returned error can't find the container with id f70a9b60a6e1a901cea28421aed7a6c4810a793aab210c515fc16ccf6dbd15b6
Nov 03 22:46:07.637353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:07.637318    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:07.637472 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:07.637396    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:07.637472 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:07.637441    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:08.328514 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:08.328476    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" event={"ID":"05719b59-3d32-4f91-b8f1-fa91e1901161","Type":"ContainerStarted","Data":"8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e"}
Nov 03 22:46:08.328772 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:08.328541    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" event={"ID":"05719b59-3d32-4f91-b8f1-fa91e1901161","Type":"ContainerStarted","Data":"f70a9b60a6e1a901cea28421aed7a6c4810a793aab210c515fc16ccf6dbd15b6"}
Nov 03 22:46:08.555012 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:08.554963    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podStartSLOduration=2.554948498 podStartE2EDuration="2.554948498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:08.505636816 +0000 UTC m=+1391.946174079" watchObservedRunningTime="2026-11-03 22:46:08.554948498 +0000 UTC m=+1391.995485755"
Nov 03 22:46:09.197718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.197650    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:09.199281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.199249    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:09.199426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.199313    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:09.542246 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.542207    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"]
Nov 03 22:46:09.546505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.546487    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.601636 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.601595    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"]
Nov 03 22:46:09.606102 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.606079    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.611373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611342    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drb84\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-kube-api-access-drb84\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.611612 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611592    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.611660 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611647    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.611762 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611748    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.611804 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611774    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.611848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611832    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.611880 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611854    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.611880 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611873    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.611949 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.611896    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.627508 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.627479    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-89xv7\""
Nov 03 22:46:09.655995 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.655966    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"]
Nov 03 22:46:09.708574 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.708538    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"]
Nov 03 22:46:09.713691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.713643    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2qq2\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-kube-api-access-p2qq2\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.713846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.713764    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.713898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.713844    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714048 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714005    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.714097 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714081    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714146 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714131    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714181 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714166    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714209    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.714281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714247    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714336 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714283    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714336 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714306    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714452 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714333    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.714648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714621    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714640    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714730    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714731    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.714814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714763    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81d99b28-c0aa-4312-9cd9-49fc570a7104-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.715028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714814    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drb84\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-kube-api-access-drb84\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.715028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714847    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.715028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.714985    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.715028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.715015    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.715219 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.715105    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.715631 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.715608    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.719271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.717642    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.720044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.720006    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.800193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.800110    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.800363 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.800251    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drb84\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-kube-api-access-drb84\") pod \"maas-default-gateway-openshift-default-7f5667f86c-d2tgj\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.816869 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.816839    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.816982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.816904    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.816982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.816941    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81d99b28-c0aa-4312-9cd9-49fc570a7104-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817078 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.816993    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817119 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817089    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817151    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817227 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817213    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817272    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817351 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817311    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2qq2\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-kube-api-access-p2qq2\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817351 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817341    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817418    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817530    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817528    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.817761 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.817739    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81d99b28-c0aa-4312-9cd9-49fc570a7104-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.819561 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.819540    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.819792 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.819777    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.862900 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.862875    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:09.898402 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.898372    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2qq2\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-kube-api-access-p2qq2\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.898771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.898753    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-5wrhg\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:09.919714 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:09.919683    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:10.198794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.198704    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:10.198794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.198759    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:10.322469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.322439    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"]
Nov 03 22:46:10.358349 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:46:10.358321    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12957ee7_e8d8_4e4d_8060_6cf8fc91c410.slice/crio-90b6802072e392d6f5c8c0c014e272c0bcaa00257cc4e1ddc6fedd4f0577627a WatchSource:0}: Error finding container 90b6802072e392d6f5c8c0c014e272c0bcaa00257cc4e1ddc6fedd4f0577627a: Status 404 returned error can't find the container with id 90b6802072e392d6f5c8c0c014e272c0bcaa00257cc4e1ddc6fedd4f0577627a
Nov 03 22:46:10.360806 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.360772    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:10.360892 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.360843    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:10.360892 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.360874    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:10.400198 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.400170    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"]
Nov 03 22:46:10.402970 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:46:10.402940    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d99b28_c0aa_4312_9cd9_49fc570a7104.slice/crio-dc96ff1667a9f6f5e92e0f297eb657da684437985280fb0c2b9fd4d4286940c0 WatchSource:0}: Error finding container dc96ff1667a9f6f5e92e0f297eb657da684437985280fb0c2b9fd4d4286940c0: Status 404 returned error can't find the container with id dc96ff1667a9f6f5e92e0f297eb657da684437985280fb0c2b9fd4d4286940c0
Nov 03 22:46:10.406276 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.406236    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:10.406373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.406325    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:10.406433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:10.406371    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:11.201707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.198059    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:11.201707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.198118    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:11.916739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.911168    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" event={"ID":"12957ee7-e8d8-4e4d-8060-6cf8fc91c410","Type":"ContainerStarted","Data":"832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4"}
Nov 03 22:46:11.916739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.911242    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" event={"ID":"12957ee7-e8d8-4e4d-8060-6cf8fc91c410","Type":"ContainerStarted","Data":"90b6802072e392d6f5c8c0c014e272c0bcaa00257cc4e1ddc6fedd4f0577627a"}
Nov 03 22:46:11.927278 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.927244    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:11.928294 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.928266    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:11.928506 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.928479    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:11.930322 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.930294    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" event={"ID":"81d99b28-c0aa-4312-9cd9-49fc570a7104","Type":"ContainerStarted","Data":"f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8"}
Nov 03 22:46:11.930500 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.930484    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" event={"ID":"81d99b28-c0aa-4312-9cd9-49fc570a7104","Type":"ContainerStarted","Data":"dc96ff1667a9f6f5e92e0f297eb657da684437985280fb0c2b9fd4d4286940c0"}
Nov 03 22:46:11.932974 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.932946    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:11.933238 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.933171    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:11.933404 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:11.933385    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:12.316695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.312465    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:12.316695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.312591    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:12.577445 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.577339    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:12.732704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.718660    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podStartSLOduration=3.718637139 podStartE2EDuration="3.718637139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:12.082218731 +0000 UTC m=+1395.522755989" watchObservedRunningTime="2026-11-03 22:46:12.718637139 +0000 UTC m=+1396.159174401"
Nov 03 22:46:12.891841 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.876957    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:12.891841 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.877023    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:12.945822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.942195    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:46:12.945822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.943036    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:12.962503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.962417    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:12.962503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:12.962494    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:13.015689 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.003514    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"]
Nov 03 22:46:13.265747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.101766    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Nov 03 22:46:13.265747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.040313    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podStartSLOduration=4.040290957 podStartE2EDuration="4.040290957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:12.628821768 +0000 UTC m=+1396.069359028" watchObservedRunningTime="2026-11-03 22:46:13.040290957 +0000 UTC m=+1396.480828216"
Nov 03 22:46:13.265747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.194282    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:13.265747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.194325    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:13.265747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.260360    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1cb89500-dockercfg-zcjwv\""
Nov 03 22:46:13.265747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.261050    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:13.265747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.261095    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:13.296859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.283987    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1cb89500\""
Nov 03 22:46:13.297281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.297233    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:13.354064 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.354020    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"]
Nov 03 22:46:13.384701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.369342    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ptjk\" (UniqueName: \"kubernetes.io/projected/bb24e377-8d98-428e-ad53-e70eb4a18e13-kube-api-access-5ptjk\") pod \"payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw\" (UID: \"bb24e377-8d98-428e-ad53-e70eb4a18e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:13.414890 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.414858    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb24e377-8d98-428e-ad53-e70eb4a18e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw\" (UID: \"bb24e377-8d98-428e-ad53-e70eb4a18e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:13.475168 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.475133    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"]
Nov 03 22:46:13.538995 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.538903    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ptjk\" (UniqueName: \"kubernetes.io/projected/bb24e377-8d98-428e-ad53-e70eb4a18e13-kube-api-access-5ptjk\") pod \"payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw\" (UID: \"bb24e377-8d98-428e-ad53-e70eb4a18e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:13.565398 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.565369    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:13.568339 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.566441    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb24e377-8d98-428e-ad53-e70eb4a18e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw\" (UID: \"bb24e377-8d98-428e-ad53-e70eb4a18e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:13.576200 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.567652    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb24e377-8d98-428e-ad53-e70eb4a18e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw\" (UID: \"bb24e377-8d98-428e-ad53-e70eb4a18e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:13.591854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.584775    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:13.591854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.585170    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:46:13.591854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.589109    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:13.625345 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.624485    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"]
Nov 03 22:46:13.658453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.658416    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ptjk\" (UniqueName: \"kubernetes.io/projected/bb24e377-8d98-428e-ad53-e70eb4a18e13-kube-api-access-5ptjk\") pod \"payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw\" (UID: \"bb24e377-8d98-428e-ad53-e70eb4a18e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:13.683699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.682074    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:13.780713 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.768808    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-plugins-config-volume\") pod \"payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6\" (UID: \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\") " pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:13.788248 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.788206    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:13.803274 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.803176    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl64v\" (UniqueName: \"kubernetes.io/projected/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-kube-api-access-nl64v\") pod \"payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6\" (UID: \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\") " pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:13.996771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.958842    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:14.161046 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.160942    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:14.161279 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.104137    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Nov 03 22:46:14.165100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.999757    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-plugins-config-volume\") pod \"payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6\" (UID: \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\") " pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:14.172884 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.172833    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl64v\" (UniqueName: \"kubernetes.io/projected/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-kube-api-access-nl64v\") pod \"payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6\" (UID: \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\") " pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:14.185748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.176829    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:46:14.203823 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:13.998133    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:14.204139 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.204088    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:14.204309 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.196245    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:14.304708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.207109    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:46:14.379687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.330688    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:14.379687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.335522    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:14.379687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.336023    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:14.379687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.336274    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:46:14.379687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.341812    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-plugins-config-volume\") pod \"payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6\" (UID: \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\") " pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:14.379687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.348621    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:14.379687 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.352875    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:14.400769 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.400708    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:14.444802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.434297    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 22:46:14.444802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.436824    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl64v\" (UniqueName: \"kubernetes.io/projected/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-kube-api-access-nl64v\") pod \"payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6\" (UID: \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\") " pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:14.645700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.634730    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:46:14.668828 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.667591    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:14.696832 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.692127    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:14.698089 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.698062    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:46:14.851142 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.851101    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": context deadline exceeded"
Nov 03 22:46:14.902941 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.889386    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:14.902941 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.889444    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:14.927871 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.927837    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:14.928157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:14.928125    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:15.058335 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.057150    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 22:46:15.212982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.179272    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:15.259891 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.259801    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:15.263805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.263767    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:15.336270 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.336224    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded"
Nov 03 22:46:15.337846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.337809    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:46:15.410014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.409974    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:15.411833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.411809    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:46:15.631583 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.631500    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:46:15.641778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.641736    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:15.702558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.692778    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Nov 03 22:46:15.956887 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:15.954986    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:16.062239 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.062201    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:16.062540 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.062493    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:16.211279 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.211188    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:16.211538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.211503    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:16.237990 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.217318    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:16.238652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.238608    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:16.238864 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.221841    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 03 22:46:16.252707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.251402    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:46:16.492782 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.221717    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:16.648278 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.553982    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Nov 03 22:46:16.649075 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.565464    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 22:46:16.654251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.569165    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:46:16.654251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.611946    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:46:16.654251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.653398    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:16.654251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.625725    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 22:46:16.668955 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:46:16.621071    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:46:16.698171 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.698130    5647 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:16.752180 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.736295    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 22:46:16.992844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.968946    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:16.995360 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:16.995324    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:17.149974 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.149943    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:17.158779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.153975    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:17.195699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.175596    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 22:46:17.195699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.175647    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"]
Nov 03 22:46:17.195699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.175698    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"]
Nov 03 22:46:17.237887 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.237740    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Nov 03 22:46:17.237887 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.237790    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Nov 03 22:46:17.863261 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.863227    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:17.863886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.863852    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:17.920701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.920155    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:17.920701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:17.920216    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:18.167633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.165300    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6" event={"ID":"fc2ae31d-4cf4-40bf-a192-f2fac423d64f","Type":"ContainerStarted","Data":"3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac"}
Nov 03 22:46:18.167633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.165375    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6" event={"ID":"fc2ae31d-4cf4-40bf-a192-f2fac423d64f","Type":"ContainerStarted","Data":"4c359c649daea0c15dfecc9bb13b1ccae4bbf020697e62f8e3bd12f169947fdd"}
Nov 03 22:46:18.167633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.165502    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:18.167633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.167431    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw" event={"ID":"bb24e377-8d98-428e-ad53-e70eb4a18e13","Type":"ContainerStarted","Data":"5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e"}
Nov 03 22:46:18.167633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.167477    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw" event={"ID":"bb24e377-8d98-428e-ad53-e70eb4a18e13","Type":"ContainerStarted","Data":"aff59b27996145f9f8ee5b06043105b4607b7169f1a1deff33f01855a653d0fa"}
Nov 03 22:46:18.167633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.167498    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:18.330700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.330029    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6" podStartSLOduration=6.330009615 podStartE2EDuration="6.330009615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:18.304038849 +0000 UTC m=+1401.744576109" watchObservedRunningTime="2026-11-03 22:46:18.330009615 +0000 UTC m=+1401.770546872"
Nov 03 22:46:18.441749 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.439661    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw" podStartSLOduration=6.439640495 podStartE2EDuration="6.439640495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:18.434131857 +0000 UTC m=+1401.874669116" watchObservedRunningTime="2026-11-03 22:46:18.439640495 +0000 UTC m=+1401.880177754"
Nov 03 22:46:18.881149 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.881095    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:18.881792 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.881754    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:18.941693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.940645    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:18.941693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:18.940812    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:19.256061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:19.255751    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:46:19.256061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:19.255822    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:46:19.863616 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:19.863585    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:19.863820 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:19.863645    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:19.920166 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:19.920132    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:19.920573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:19.920194    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:20.007376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:20.007338    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:20.007724 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:20.007698    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:20.008412 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:20.008393    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:46:20.905695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:20.897857    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Nov 03 22:46:20.905695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:20.897915    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Nov 03 22:46:20.925708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:20.922108    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Nov 03 22:46:20.925708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:20.922190    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Nov 03 22:46:22.149295 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:22.149247    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:22.150249 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:22.150224    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:22.151146 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:22.151126    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 22:46:22.400442 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:22.400364    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:22.400747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:22.400710    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:22.401530 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:22.401511    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 22:46:24.685033 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.684999    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"]
Nov 03 22:46:24.689595 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.689570    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.776230 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776194    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.776424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776265    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.776424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776371    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zm9t\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-kube-api-access-4zm9t\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.776424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776411    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.776580 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776440    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.776644 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776627    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.776746 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776662    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.776943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776926    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f55b8f-0627-455d-a540-3e02a6d25c01-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.777012 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.776983    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.955539    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.955588    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.955839    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f55b8f-0627-455d-a540-3e02a6d25c01-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.955891    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.955931    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.955984    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.956034    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zm9t\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-kube-api-access-4zm9t\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.956079    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.965776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.956106    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.997693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.997092    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.998515 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.998486    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.999233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.999209    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:24.999813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:24.999794    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:25.000517 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.000497    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f55b8f-0627-455d-a540-3e02a6d25c01-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:25.001575 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.000909    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:25.080146 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.080100    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:25.115692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.115029    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"]
Nov 03 22:46:25.181692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.174841    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zm9t\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-kube-api-access-4zm9t\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:25.195839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.195788    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-np57c\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:25.304530 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.304442    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:25.868657 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.864909    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:25.868657 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.865000    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:25.868657 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.865044    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:46:25.913036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:25.912996    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"]
Nov 03 22:46:26.263928 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:26.263882    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" event={"ID":"66f55b8f-0627-455d-a540-3e02a6d25c01","Type":"ContainerStarted","Data":"92432074c9316d7ffd93756d0965f9e1bc050b43adb38dfb74aca7c8f75944db"}
Nov 03 22:46:26.264101 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:26.263940    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" event={"ID":"66f55b8f-0627-455d-a540-3e02a6d25c01","Type":"ContainerStarted","Data":"975ecf23617e74e5a1b93fa848aea03941e2f791a5c174dc1f330051c0ea816d"}
Nov 03 22:46:26.305136 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:26.305103    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:26.388378 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:26.388325    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podStartSLOduration=2.388311075 podStartE2EDuration="2.388311075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:46:26.384189652 +0000 UTC m=+1409.824726913" watchObservedRunningTime="2026-11-03 22:46:26.388311075 +0000 UTC m=+1409.828848398"
Nov 03 22:46:27.306229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:27.306197    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-np57c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 03 22:46:27.306623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:27.306250    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 03 22:46:28.305530 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:28.305498    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-np57c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 03 22:46:28.305734 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:28.305561    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 03 22:46:29.205402 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:29.205366    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:46:29.206151 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:29.206125    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:46:29.305586 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:29.305551    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-np57c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 03 22:46:29.305783 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:29.305612    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 03 22:46:30.305350 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:30.305314    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-np57c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 03 22:46:30.305786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:30.305377    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 03 22:46:31.313783 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:31.309216    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-np57c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 03 22:46:31.313783 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:31.309276    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 03 22:46:32.305002 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:32.304942    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-np57c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 03 22:46:32.305229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:32.305011    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 03 22:46:33.305749 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:33.305711    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-np57c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 03 22:46:33.306192 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:33.305785    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 03 22:46:34.558865 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:34.558834    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:34.559332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:34.559258    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:34.560250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:34.560230    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 22:46:37.415876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:37.415843    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:46:43.597087 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:43.597044    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"]
Nov 03 22:46:43.702816 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:43.702763    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"]
Nov 03 22:46:46.043787 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:46:46.043749    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:46:48.598360 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.598321    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz" podUID="c23f52be-ec70-459e-b6a0-3c599e68584f" containerName="payload-processing" containerID="cri-o://5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7" gracePeriod=25
Nov 03 22:46:48.704303 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.704270    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x" podUID="f15fb519-4d3b-4eed-b0b6-c5d76e4942b3" containerName="payload-pre-processing" containerID="cri-o://e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42" gracePeriod=25
Nov 03 22:46:48.882316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.882290    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:46:48.942094 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.942065    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c23f52be-ec70-459e-b6a0-3c599e68584f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23f52be-ec70-459e-b6a0-3c599e68584f-plugins-config-volume\") pod \"c23f52be-ec70-459e-b6a0-3c599e68584f\" (UID: \"c23f52be-ec70-459e-b6a0-3c599e68584f\") "
Nov 03 22:46:48.942286 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.942111    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c23f52be-ec70-459e-b6a0-3c599e68584f-kube-api-access-ltqm9\" (UniqueName: \"kubernetes.io/projected/c23f52be-ec70-459e-b6a0-3c599e68584f-kube-api-access-ltqm9\") pod \"c23f52be-ec70-459e-b6a0-3c599e68584f\" (UID: \"c23f52be-ec70-459e-b6a0-3c599e68584f\") "
Nov 03 22:46:48.942470 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.942447    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23f52be-ec70-459e-b6a0-3c599e68584f-plugins-config-volume" pod "c23f52be-ec70-459e-b6a0-3c599e68584f" (UID: "c23f52be-ec70-459e-b6a0-3c599e68584f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:46:48.942945 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.942923    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23f52be-ec70-459e-b6a0-3c599e68584f-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:46:48.946197 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.946170    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23f52be-ec70-459e-b6a0-3c599e68584f-kube-api-access-ltqm9" pod "c23f52be-ec70-459e-b6a0-3c599e68584f" (UID: "c23f52be-ec70-459e-b6a0-3c599e68584f"). InnerVolumeSpecName "kube-api-access-ltqm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:46:48.976127 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:48.976101    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:46:49.043634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.043602    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-plugins-config-volume\") pod \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\" (UID: \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\") "
Nov 03 22:46:49.044057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.043897    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-kube-api-access-ldwmc\" (UniqueName: \"kubernetes.io/projected/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-kube-api-access-ldwmc\") pod \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\" (UID: \"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3\") "
Nov 03 22:46:49.044134 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.044054    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-plugins-config-volume" pod "f15fb519-4d3b-4eed-b0b6-c5d76e4942b3" (UID: "f15fb519-4d3b-4eed-b0b6-c5d76e4942b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:46:49.044585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.044559    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:46:49.044585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.044584    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltqm9\" (UniqueName: \"kubernetes.io/projected/c23f52be-ec70-459e-b6a0-3c599e68584f-kube-api-access-ltqm9\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:46:49.046529 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.046478    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-kube-api-access-ldwmc" pod "f15fb519-4d3b-4eed-b0b6-c5d76e4942b3" (UID: "f15fb519-4d3b-4eed-b0b6-c5d76e4942b3"). InnerVolumeSpecName "kube-api-access-ldwmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:46:49.146367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.146280    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldwmc\" (UniqueName: \"kubernetes.io/projected/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3-kube-api-access-ldwmc\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:46:49.640026 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.639988    5647 generic.go:356] "Generic (PLEG): container finished" podID="f15fb519-4d3b-4eed-b0b6-c5d76e4942b3" containerID="e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42" exitCode=0
Nov 03 22:46:49.640992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.640038    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x" event={"ID":"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3","Type":"ContainerDied","Data":"e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42"}
Nov 03 22:46:49.640992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.640087    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"
Nov 03 22:46:49.642554 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.642524    5647 generic.go:356] "Generic (PLEG): container finished" podID="c23f52be-ec70-459e-b6a0-3c599e68584f" containerID="5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7" exitCode=0
Nov 03 22:46:49.642767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.642730    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"
Nov 03 22:46:49.645287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.644766    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x" event={"ID":"f15fb519-4d3b-4eed-b0b6-c5d76e4942b3","Type":"ContainerDied","Data":"6b66f2e54dc967521f678cd16a6c513c3a356336377921fbceb103c2b0fcf09b"}
Nov 03 22:46:49.645287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.644834    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz" event={"ID":"c23f52be-ec70-459e-b6a0-3c599e68584f","Type":"ContainerDied","Data":"5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7"}
Nov 03 22:46:49.645287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.644861    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz" event={"ID":"c23f52be-ec70-459e-b6a0-3c599e68584f","Type":"ContainerDied","Data":"284fe5bddf63fb1e8b5873d011485c916ceed76a411ff036d7774a4191be4f49"}
Nov 03 22:46:49.645287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.644891    5647 scope.go:122] "RemoveContainer" containerID="e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42"
Nov 03 22:46:49.773062 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.773032    5647 scope.go:122] "RemoveContainer" containerID="e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42"
Nov 03 22:46:49.774472 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:46:49.774450    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42\": container with ID starting with e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42 not found: ID does not exist" containerID="e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42"
Nov 03 22:46:49.777303 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.774596    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42"} err="failed to get container status \"e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42\": rpc error: code = NotFound desc = could not find container \"e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42\": container with ID starting with e95d0866922179c59d24b751445e92b4fe5a27244e57dd77d6b8d31941435b42 not found: ID does not exist"
Nov 03 22:46:49.777440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.777303    5647 scope.go:122] "RemoveContainer" containerID="5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7"
Nov 03 22:46:49.789039 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.789019    5647 scope.go:122] "RemoveContainer" containerID="5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7"
Nov 03 22:46:49.789351 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:46:49.789328    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7\": container with ID starting with 5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7 not found: ID does not exist" containerID="5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7"
Nov 03 22:46:49.789403 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.789353    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7"} err="failed to get container status \"5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7\": rpc error: code = NotFound desc = could not find container \"5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7\": container with ID starting with 5dc17f2e9cc8d7012a5c4480a22049a70efd08abe196fe22413aabc58f7ad1e7 not found: ID does not exist"
Nov 03 22:46:49.893439 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.893309    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"]
Nov 03 22:46:49.897170 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:49.897147    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz"]
Nov 03 22:46:50.005555 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:50.005515    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"]
Nov 03 22:46:50.009494 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:50.009469    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x"]
Nov 03 22:46:51.045318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:51.044410    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c23f52be-ec70-459e-b6a0-3c599e68584f" path="/var/lib/kubelet/pods/c23f52be-ec70-459e-b6a0-3c599e68584f/volumes"
Nov 03 22:46:51.063754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:46:51.058071    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f15fb519-4d3b-4eed-b0b6-c5d76e4942b3" path="/var/lib/kubelet/pods/f15fb519-4d3b-4eed-b0b6-c5d76e4942b3/volumes"
Nov 03 22:47:03.369520 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:03.369440    5647 generic.go:356] "Generic (PLEG): container finished" podID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerID="038aa33460d32934ec9d304003206b8cfc410fb39ef37c460c44a75dad379fe6" exitCode=0
Nov 03 22:47:03.369520 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:03.369506    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" event={"ID":"daa54680-0cbb-48f4-b08d-f391d5c94158","Type":"ContainerDied","Data":"038aa33460d32934ec9d304003206b8cfc410fb39ef37c460c44a75dad379fe6"}
Nov 03 22:47:03.370052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:03.370034    5647 scope.go:122] "RemoveContainer" containerID="038aa33460d32934ec9d304003206b8cfc410fb39ef37c460c44a75dad379fe6"
Nov 03 22:47:03.371700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:03.371648    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:03.371805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:03.371732    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:03.371805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:03.371765    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:04.402795 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:04.402759    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" event={"ID":"daa54680-0cbb-48f4-b08d-f391d5c94158","Type":"ContainerStarted","Data":"298e1db76895c7b6f74ec7a51d93230c2f3ec2ecf8af00812ca4a4c6b456cc55"}
Nov 03 22:47:04.610381 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:04.610345    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:47:04.909836 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:04.909789    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:47:04.910866 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:04.910842    5647 patch_prober.go:29] interesting pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:47:04.910978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:04.910890    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:47:05.909716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:05.909659    5647 patch_prober.go:29] interesting pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:47:05.910117 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:05.909764    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:47:06.910028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:06.909940    5647 patch_prober.go:29] interesting pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:47:06.910028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:06.909998    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:47:07.910020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:07.909981    5647 patch_prober.go:29] interesting pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:47:07.910454 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:07.910050    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:47:08.909829 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:08.909784    5647 patch_prober.go:29] interesting pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 03 22:47:08.910054 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:08.909848    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 03 22:47:09.913361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:09.913332    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:47:09.914500 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:09.914482    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:47:16.044340 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:47:16.044303    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:47:18.025691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.000486    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"]
Nov 03 22:47:18.040700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.027429    5647 cpu_manager.go:414] "containerMap: removing container" podUID="c23f52be-ec70-459e-b6a0-3c599e68584f" containerName="payload-processing"
Nov 03 22:47:18.040958 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.040930    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c23f52be-ec70-459e-b6a0-3c599e68584f" containerName="payload-processing"
Nov 03 22:47:18.041125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.041112    5647 cpu_manager.go:414] "containerMap: removing container" podUID="f15fb519-4d3b-4eed-b0b6-c5d76e4942b3" containerName="payload-pre-processing"
Nov 03 22:47:18.041413 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.041395    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f15fb519-4d3b-4eed-b0b6-c5d76e4942b3" containerName="payload-pre-processing"
Nov 03 22:47:18.041894 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.041875    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="c23f52be-ec70-459e-b6a0-3c599e68584f" containerName="payload-processing"
Nov 03 22:47:18.048702 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.042014    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="f15fb519-4d3b-4eed-b0b6-c5d76e4942b3" containerName="payload-pre-processing"
Nov 03 22:47:18.254838 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.254801    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"]
Nov 03 22:47:18.255246 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.255140    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.315038 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.314788    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-26b8d994-openshift-default-dockercfg-c7mj8\""
Nov 03 22:47:18.465134 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.465089    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-token\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.466028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.465999    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-envoy\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.466564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.466541    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-socket\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.466794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.466761    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-credential-socket\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.470307 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.470246    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-data\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.470707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.470654    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-podinfo\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.470912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.470895    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-certs\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.471085 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.471068    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istiod-ca-cert\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.471283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.471266    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jcsg\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-kube-api-access-4jcsg\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.578754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.578381    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-credential-socket\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.578754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.578460    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-data\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.578754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.578548    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-podinfo\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.578754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.578601    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-certs\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.578754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.578699    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istiod-ca-cert\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.578754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.578757    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jcsg\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-kube-api-access-4jcsg\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.584999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.578818    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-token\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.584999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.578853    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-envoy\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.584999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.579104    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-socket\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.584999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.579644    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-socket\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.584999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.580305    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-credential-socket\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.584999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.580685    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-data\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.584999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.581584    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-certs\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.584999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.583700    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istiod-ca-cert\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.608343 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.608307    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-podinfo\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.625996 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.625831    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-envoy\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.662741 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.660767    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jcsg\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-kube-api-access-4jcsg\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.672158 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.672059    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-token\") pod \"e2e-ait-26b8d994-openshift-default-78f469648f-4psc8\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") " pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:18.897608 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:18.897521    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:19.335915 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:19.335883    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"]
Nov 03 22:47:19.336594 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:47:19.336563    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39c73d8_a10c_44f5_98d8_a0ccf1a4f1fa.slice/crio-0188be2941a3466361c925687f9442e12386b717d54ac2ed3fbb452629f51d37 WatchSource:0}: Error finding container 0188be2941a3466361c925687f9442e12386b717d54ac2ed3fbb452629f51d37: Status 404 returned error can't find the container with id 0188be2941a3466361c925687f9442e12386b717d54ac2ed3fbb452629f51d37
Nov 03 22:47:19.339372 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:19.339335    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:19.339461 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:19.339404    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:19.339461 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:19.339433    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:19.666196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:19.666108    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" event={"ID":"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa","Type":"ContainerStarted","Data":"8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822"}
Nov 03 22:47:19.666196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:19.666164    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" event={"ID":"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa","Type":"ContainerStarted","Data":"0188be2941a3466361c925687f9442e12386b717d54ac2ed3fbb452629f51d37"}
Nov 03 22:47:19.785142 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:19.784941    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podStartSLOduration=2.784920604 podStartE2EDuration="2.784920604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:47:19.784190719 +0000 UTC m=+1463.224728001" watchObservedRunningTime="2026-11-03 22:47:19.784920604 +0000 UTC m=+1463.225457864"
Nov 03 22:47:19.917752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:19.911967    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:20.904717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:20.904684    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:21.679703 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:21.679636    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:21.680625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:21.680602    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:47:22.545409 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:22.545249    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:47:23.223961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:23.223920    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"]
Nov 03 22:47:23.224207 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:23.224181    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy" containerID="cri-o://298e1db76895c7b6f74ec7a51d93230c2f3ec2ecf8af00812ca4a4c6b456cc55" gracePeriod=30
Nov 03 22:47:24.917300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:24.917262    5647 patch_prober.go:29] interesting pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:47:24.917845 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:24.917814    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:47:25.914800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:25.906106    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"]
Nov 03 22:47:26.002871 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.002832    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"]
Nov 03 22:47:26.003640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.003607    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:26.125095 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.123322    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-26b8d994\""
Nov 03 22:47:26.125844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.125376    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6e24077-c448-45b0-bb3c-be6f41cedc5f-plugins-config-volume\") pod \"payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch\" (UID: \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\") " pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:26.125844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.125548    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xl2t\" (UniqueName: \"kubernetes.io/projected/b6e24077-c448-45b0-bb3c-be6f41cedc5f-kube-api-access-8xl2t\") pod \"payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch\" (UID: \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\") " pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:26.150221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.150183    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-26b8d994-dockercfg-vwfqn\""
Nov 03 22:47:26.212598 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.199684    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"]
Nov 03 22:47:26.243697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.218844    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:26.331697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.248528    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6e24077-c448-45b0-bb3c-be6f41cedc5f-plugins-config-volume\") pod \"payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch\" (UID: \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\") " pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:26.331697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.248728    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xl2t\" (UniqueName: \"kubernetes.io/projected/b6e24077-c448-45b0-bb3c-be6f41cedc5f-kube-api-access-8xl2t\") pod \"payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch\" (UID: \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\") " pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:26.352698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.332215    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6e24077-c448-45b0-bb3c-be6f41cedc5f-plugins-config-volume\") pod \"payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch\" (UID: \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\") " pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:26.534799 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.509545    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40dea850-f578-48f8-b11c-5a47e40699d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9\" (UID: \"40dea850-f578-48f8-b11c-5a47e40699d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:26.534799 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.509744    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz5pg\" (UniqueName: \"kubernetes.io/projected/40dea850-f578-48f8-b11c-5a47e40699d3-kube-api-access-nz5pg\") pod \"payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9\" (UID: \"40dea850-f578-48f8-b11c-5a47e40699d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:26.625178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.625126    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40dea850-f578-48f8-b11c-5a47e40699d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9\" (UID: \"40dea850-f578-48f8-b11c-5a47e40699d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:26.625648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.625613    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz5pg\" (UniqueName: \"kubernetes.io/projected/40dea850-f578-48f8-b11c-5a47e40699d3-kube-api-access-nz5pg\") pod \"payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9\" (UID: \"40dea850-f578-48f8-b11c-5a47e40699d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:26.643272 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.643227    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40dea850-f578-48f8-b11c-5a47e40699d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9\" (UID: \"40dea850-f578-48f8-b11c-5a47e40699d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:26.800735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.800621    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xl2t\" (UniqueName: \"kubernetes.io/projected/b6e24077-c448-45b0-bb3c-be6f41cedc5f-kube-api-access-8xl2t\") pod \"payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch\" (UID: \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\") " pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:26.806025 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.805988    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz5pg\" (UniqueName: \"kubernetes.io/projected/40dea850-f578-48f8-b11c-5a47e40699d3-kube-api-access-nz5pg\") pod \"payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9\" (UID: \"40dea850-f578-48f8-b11c-5a47e40699d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:26.817870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.817811    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:26.872272 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:26.872194    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"]
Nov 03 22:47:27.105823 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:27.103083    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:27.811487 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:27.810538    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"]
Nov 03 22:47:27.873966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:27.853078    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"]
Nov 03 22:47:28.135252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:28.135162    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch" event={"ID":"b6e24077-c448-45b0-bb3c-be6f41cedc5f","Type":"ContainerStarted","Data":"37a9f8a01db0724fe6ef96d0fa9f5beb0e270aaaafc2750c78463cee2e675392"}
Nov 03 22:47:28.692701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:28.670798    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:28.692701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:28.671235    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:28.845684 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:28.845556    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:29.327710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.308839    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:47:29.505998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.474894    5647 generic.go:356] "Generic (PLEG): container finished" podID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerID="298e1db76895c7b6f74ec7a51d93230c2f3ec2ecf8af00812ca4a4c6b456cc55" exitCode=0
Nov 03 22:47:29.505998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.475072    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" event={"ID":"daa54680-0cbb-48f4-b08d-f391d5c94158","Type":"ContainerDied","Data":"298e1db76895c7b6f74ec7a51d93230c2f3ec2ecf8af00812ca4a4c6b456cc55"}
Nov 03 22:47:29.505998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.475132    5647 scope.go:122] "RemoveContainer" containerID="038aa33460d32934ec9d304003206b8cfc410fb39ef37c460c44a75dad379fe6"
Nov 03 22:47:29.745464 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.745424    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:29.866036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.865934    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9" event={"ID":"40dea850-f578-48f8-b11c-5a47e40699d3","Type":"ContainerStarted","Data":"2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49"}
Nov 03 22:47:29.866405 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.866278    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9" event={"ID":"40dea850-f578-48f8-b11c-5a47e40699d3","Type":"ContainerStarted","Data":"af76fb4da2c2cba02a9febe6dbd251440347aed6378b8d5b73c4b50b05f56877"}
Nov 03 22:47:29.868169 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.868150    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:29.875351 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.875317    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch" event={"ID":"b6e24077-c448-45b0-bb3c-be6f41cedc5f","Type":"ContainerStarted","Data":"2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15"}
Nov 03 22:47:29.876503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.876475    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:29.883033 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.883005    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:47:29.995694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:29.995230    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.079263    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-certs\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.079415    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-data\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.079541    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-envoy\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.079601    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-kube-api-access-pftnk\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-kube-api-access-pftnk\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.079733    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-podinfo\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.079815    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-token\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-token\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.079881    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daa54680-0cbb-48f4-b08d-f391d5c94158-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa54680-0cbb-48f4-b08d-f391d5c94158-istiod-ca-cert\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.079971    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-credential-socket\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.082905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.080052    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-socket\") pod \"daa54680-0cbb-48f4-b08d-f391d5c94158\" (UID: \"daa54680-0cbb-48f4-b08d-f391d5c94158\") "
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.093414    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-certs" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.093827    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-data" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.094229    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daa54680-0cbb-48f4-b08d-f391d5c94158-istiod-ca-cert" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.094801    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-credential-socket" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.095100    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-socket" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.096778    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.096804    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.096820    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.096850    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daa54680-0cbb-48f4-b08d-f391d5c94158-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.098872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.096865    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.152765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.148132    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-token" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:47:30.171417 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.171364    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-podinfo" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:47:30.189588 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.189521    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9" podStartSLOduration=4.189497023 podStartE2EDuration="4.189497023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:47:30.060091736 +0000 UTC m=+1473.500628996" watchObservedRunningTime="2026-11-03 22:47:30.189497023 +0000 UTC m=+1473.630034283"
Nov 03 22:47:30.223431 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.196428    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-envoy" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:47:30.223934 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.223289    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.224207 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.224188    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.329712 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.325764    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 03 22:47:30.329712 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.326501    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:47:30.369271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.356148    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-kube-api-access-pftnk" pod "daa54680-0cbb-48f4-b08d-f391d5c94158" (UID: "daa54680-0cbb-48f4-b08d-f391d5c94158"). InnerVolumeSpecName "kube-api-access-pftnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:47:30.369271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.357265    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daa54680-0cbb-48f4-b08d-f391d5c94158-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.369271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.357290    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pftnk\" (UniqueName: \"kubernetes.io/projected/daa54680-0cbb-48f4-b08d-f391d5c94158-kube-api-access-pftnk\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:47:30.633704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.572567    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:30.727944 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.726124    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:30.829191 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.746876    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:30.829191 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.751943    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:47:30.829191 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.818486    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch" podStartSLOduration=5.818461646 podStartE2EDuration="5.818461646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:47:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:47:30.348455151 +0000 UTC m=+1473.788992411" watchObservedRunningTime="2026-11-03 22:47:30.818461646 +0000 UTC m=+1474.258998906"
Nov 03 22:47:30.833841 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.833709    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:47:30.835638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.835306    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 22:47:30.836520 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.836498    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 22:47:30.915880 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:47:30.914182    5647 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa54680_0cbb_48f4_b08d_f391d5c94158.slice\": RecentStats: unable to find data in memory cache]"
Nov 03 22:47:30.915880 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:47:30.914256    5647 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa54680_0cbb_48f4_b08d_f391d5c94158.slice\": RecentStats: unable to find data in memory cache]"
Nov 03 22:47:30.925919 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.925857    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"
Nov 03 22:47:30.926160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.926135    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj" event={"ID":"daa54680-0cbb-48f4-b08d-f391d5c94158","Type":"ContainerDied","Data":"c4a590cc8cee75296ddcf012e75df45ee4cba80fef3003d511fb292ad3283a9d"}
Nov 03 22:47:30.926324 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:30.926298    5647 scope.go:122] "RemoveContainer" containerID="298e1db76895c7b6f74ec7a51d93230c2f3ec2ecf8af00812ca4a4c6b456cc55"
Nov 03 22:47:31.144807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:31.144770    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"]
Nov 03 22:47:31.148963 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:31.148933    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj"]
Nov 03 22:47:32.972699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:32.966975    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" path="/var/lib/kubelet/pods/daa54680-0cbb-48f4-b08d-f391d5c94158/volumes"
Nov 03 22:47:32.972699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:32.967457    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"]
Nov 03 22:47:33.016074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.000009    5647 cpu_manager.go:414] "containerMap: removing container" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy"
Nov 03 22:47:33.016074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.000040    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy"
Nov 03 22:47:33.067770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.020254    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy"
Nov 03 22:47:33.067770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.020310    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy"
Nov 03 22:47:33.247985 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.247895    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"]
Nov 03 22:47:33.248661 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.248608    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:33.258916 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.258864    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"]
Nov 03 22:47:33.287695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.260095    5647 cpu_manager.go:414] "containerMap: removing container" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy"
Nov 03 22:47:33.287695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.260120    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daa54680-0cbb-48f4-b08d-f391d5c94158" containerName="istio-proxy"
Nov 03 22:47:33.403522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.346247    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grfbx\" (UniqueName: \"kubernetes.io/projected/1f9bd96a-a1b0-495b-aada-b8e6595686c4-kube-api-access-grfbx\") pod \"payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b\" (UID: \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:33.403522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.346388    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9bd96a-a1b0-495b-aada-b8e6595686c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b\" (UID: \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:33.403522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.372904    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"]
Nov 03 22:47:33.403522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.373029    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:33.403522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.373787    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-9d7aa682\""
Nov 03 22:47:33.403522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.373893    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-9d7aa682-dockercfg-94gxt\""
Nov 03 22:47:33.481697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.472057    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grfbx\" (UniqueName: \"kubernetes.io/projected/1f9bd96a-a1b0-495b-aada-b8e6595686c4-kube-api-access-grfbx\") pod \"payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b\" (UID: \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:33.481697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.472165    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvdsn\" (UniqueName: \"kubernetes.io/projected/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-kube-api-access-mvdsn\") pod \"payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp\" (UID: \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\") " pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:33.481697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.472243    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9bd96a-a1b0-495b-aada-b8e6595686c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b\" (UID: \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:33.481697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.472306    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-plugins-config-volume\") pod \"payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp\" (UID: \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\") " pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:33.514683 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.514574    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9bd96a-a1b0-495b-aada-b8e6595686c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b\" (UID: \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:33.574374 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.574339    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grfbx\" (UniqueName: \"kubernetes.io/projected/1f9bd96a-a1b0-495b-aada-b8e6595686c4-kube-api-access-grfbx\") pod \"payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b\" (UID: \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:33.588311 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.588275    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvdsn\" (UniqueName: \"kubernetes.io/projected/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-kube-api-access-mvdsn\") pod \"payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp\" (UID: \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\") " pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:33.589080 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.589054    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-plugins-config-volume\") pod \"payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp\" (UID: \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\") " pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:33.593726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.590557    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-plugins-config-volume\") pod \"payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp\" (UID: \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\") " pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:33.736536 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.736460    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvdsn\" (UniqueName: \"kubernetes.io/projected/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-kube-api-access-mvdsn\") pod \"payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp\" (UID: \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\") " pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:33.772788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.772319    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:33.915371 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:33.915315    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:34.186269 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:34.186226    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"]
Nov 03 22:47:34.186951 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:47:34.186710    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9bd96a_a1b0_495b_aada_b8e6595686c4.slice/crio-f3b1d67b53e58a3c5d8f6a2733b8817ffbb9e873ec4184f5a6460eab346e8784 WatchSource:0}: Error finding container f3b1d67b53e58a3c5d8f6a2733b8817ffbb9e873ec4184f5a6460eab346e8784: Status 404 returned error can't find the container with id f3b1d67b53e58a3c5d8f6a2733b8817ffbb9e873ec4184f5a6460eab346e8784
Nov 03 22:47:34.312134 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:34.312103    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"]
Nov 03 22:47:34.312516 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:47:34.312489    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ee82ba_6c08_4c3d_9792_a8654e8c69b8.slice/crio-d52ee76b657c96e99f0983e92bac70bc6c0ff5eb369ab5537ce0dee7b021e677 WatchSource:0}: Error finding container d52ee76b657c96e99f0983e92bac70bc6c0ff5eb369ab5537ce0dee7b021e677: Status 404 returned error can't find the container with id d52ee76b657c96e99f0983e92bac70bc6c0ff5eb369ab5537ce0dee7b021e677
Nov 03 22:47:35.080485 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:35.080445    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b" event={"ID":"1f9bd96a-a1b0-495b-aada-b8e6595686c4","Type":"ContainerStarted","Data":"30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11"}
Nov 03 22:47:35.080724 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:35.080522    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:35.080724 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:35.080542    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b" event={"ID":"1f9bd96a-a1b0-495b-aada-b8e6595686c4","Type":"ContainerStarted","Data":"f3b1d67b53e58a3c5d8f6a2733b8817ffbb9e873ec4184f5a6460eab346e8784"}
Nov 03 22:47:35.082401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:35.082374    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp" event={"ID":"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8","Type":"ContainerStarted","Data":"1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7"}
Nov 03 22:47:35.082504 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:35.082424    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp" event={"ID":"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8","Type":"ContainerStarted","Data":"d52ee76b657c96e99f0983e92bac70bc6c0ff5eb369ab5537ce0dee7b021e677"}
Nov 03 22:47:35.082504 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:35.082449    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:35.201349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:35.201201    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b" podStartSLOduration=3.201181674 podStartE2EDuration="3.201181674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:47:35.194550991 +0000 UTC m=+1478.635088247" watchObservedRunningTime="2026-11-03 22:47:35.201181674 +0000 UTC m=+1478.641718936"
Nov 03 22:47:35.312247 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:35.312192    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp" podStartSLOduration=3.312178029 podStartE2EDuration="3.312178029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:47:35.308597901 +0000 UTC m=+1478.749135158" watchObservedRunningTime="2026-11-03 22:47:35.312178029 +0000 UTC m=+1478.752715287"
Nov 03 22:47:39.117846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:39.117809    5647 generic.go:356] "Generic (PLEG): container finished" podID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerID="b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84" exitCode=0
Nov 03 22:47:39.118329 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:39.117879    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" event={"ID":"9daf04a2-44f1-4659-b8ce-ad117f28527d","Type":"ContainerDied","Data":"b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84"}
Nov 03 22:47:39.118392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:39.118377    5647 scope.go:122] "RemoveContainer" containerID="b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84"
Nov 03 22:47:39.120160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:39.120132    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:39.120211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:39.120199    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:39.120251 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:39.120231    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:40.128508 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:40.128470    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" event={"ID":"9daf04a2-44f1-4659-b8ce-ad117f28527d","Type":"ContainerStarted","Data":"764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320"}
Nov 03 22:47:40.936634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:40.936602    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:47:40.936874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:40.936844    5647 patch_prober.go:29] interesting pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:47:40.937226 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:40.936919    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 03 22:47:41.936906 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:41.936868    5647 patch_prober.go:29] interesting pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:47:41.937385 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:41.936928    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 03 22:47:41.937385 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:41.937237    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:47:41.937609 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:41.937593    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:47:42.937188 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:42.937148    5647 patch_prober.go:29] interesting pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:47:42.937584 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:42.937206    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 03 22:47:43.937314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:43.937273    5647 patch_prober.go:29] interesting pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:47:43.937796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:43.937351    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 03 22:47:44.937060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:44.937021    5647 patch_prober.go:29] interesting pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:47:44.937256 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:44.937096    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 03 22:47:45.937446 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:45.937412    5647 patch_prober.go:29] interesting pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:47:45.937874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:45.937464    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 03 22:47:46.047333 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:47:46.047277    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:47:46.095208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:46.095178    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:47:46.095384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:46.095250    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:47:46.264964 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:46.264926    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:47:46.941575 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:46.941546    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:47:47.186412 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:47.186380    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:47:48.537156 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.537116    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"]
Nov 03 22:47:48.565142 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.565110    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.622891 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.622059    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-c93e70-openshift-default-dockercfg-hqqtc\""
Nov 03 22:47:48.706696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.698244    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"]
Nov 03 22:47:48.788303 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.788189    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-certs\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.789348 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.789296    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-istio-token\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.789491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.789369    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk55j\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-kube-api-access-kk55j\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.789637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.789498    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-credential-socket\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.789637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.789585    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-envoy\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.789637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.789635    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30be5938-7f33-495a-8537-a012b282e83a-istiod-ca-cert\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.790064 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.789686    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30be5938-7f33-495a-8537-a012b282e83a-istio-podinfo\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.790064 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.789713    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-data\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.790064 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.789839    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-socket\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.903953    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-istio-token\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.904094    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk55j\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-kube-api-access-kk55j\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.904242    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-credential-socket\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.904368    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-envoy\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.904435    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30be5938-7f33-495a-8537-a012b282e83a-istiod-ca-cert\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.904468    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30be5938-7f33-495a-8537-a012b282e83a-istio-podinfo\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.904498    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-data\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.904708    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-socket\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.904816    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-certs\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.905844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.905391    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-certs\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.906647 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.906565    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-data\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.909273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.906837    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-credential-socket\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.909273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.908000    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-socket\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:48.909273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:48.908645    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30be5938-7f33-495a-8537-a012b282e83a-istiod-ca-cert\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:49.049396 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:49.049157    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30be5938-7f33-495a-8537-a012b282e83a-istio-podinfo\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:49.058138 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:49.057727    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-envoy\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:49.089833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:49.089747    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk55j\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-kube-api-access-kk55j\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:49.321035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:49.320919    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-istio-token\") pod \"e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") " pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:49.510086 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:49.510046    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:50.107286 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:50.107258    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"]
Nov 03 22:47:50.108006 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:47:50.107972    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30be5938_7f33_495a_8537_a012b282e83a.slice/crio-e02ba448dbe67bb4e6cc38e89737507c615fcecd53cc626568aca793c4e9084e WatchSource:0}: Error finding container e02ba448dbe67bb4e6cc38e89737507c615fcecd53cc626568aca793c4e9084e: Status 404 returned error can't find the container with id e02ba448dbe67bb4e6cc38e89737507c615fcecd53cc626568aca793c4e9084e
Nov 03 22:47:50.111481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:50.111442    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:50.111570 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:50.111528    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:50.111611 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:50.111568    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:47:50.419398 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:50.419312    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" event={"ID":"30be5938-7f33-495a-8537-a012b282e83a","Type":"ContainerStarted","Data":"353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9"}
Nov 03 22:47:50.419398 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:50.419367    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" event={"ID":"30be5938-7f33-495a-8537-a012b282e83a","Type":"ContainerStarted","Data":"e02ba448dbe67bb4e6cc38e89737507c615fcecd53cc626568aca793c4e9084e"}
Nov 03 22:47:50.510936 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:50.510900    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:50.568165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:50.568106    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podStartSLOduration=2.56809212 podStartE2EDuration="2.56809212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:47:50.565147701 +0000 UTC m=+1494.005684961" watchObservedRunningTime="2026-11-03 22:47:50.56809212 +0000 UTC m=+1494.008629379"
Nov 03 22:47:51.516662 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:51.516633    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:52.435848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:52.435809    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:52.436643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:52.436625    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:47:56.915130 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:56.915102    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:47:56.921288 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:47:56.921267    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:48:08.415826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:08.415788    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"]
Nov 03 22:48:08.533723 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:08.533683    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"]
Nov 03 22:48:09.516410 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.516372    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"]
Nov 03 22:48:09.522299 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.522276    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.575694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.575640    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-26b8d994-serving-cert\""
Nov 03 22:48:09.581726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.581684    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"]
Nov 03 22:48:09.614454 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.614410    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.614745 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.614714    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-metrics-certs\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.614889 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.614863    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-openshift-service-ca\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.615071 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.615053    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdnvr\" (UniqueName: \"kubernetes.io/projected/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-kube-api-access-qdnvr\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.716773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.716735    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.716977 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:48:09.716879    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-serving-cert: secret "maas-api-e2e-ait-26b8d994-serving-cert" not found
Nov 03 22:48:09.716977 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.716914    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-metrics-certs\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.716977 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:48:09.716936    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls podName:a2122b1d-bbd5-4528-ba52-bcc9fb43897b nodeName:}" failed. No retries permitted until 2026-11-03 22:48:10.216919036 +0000 UTC m=+1513.657456273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls") pod "maas-api-e2e-ait-26b8d994-b5977797-2q4bq" (UID: "a2122b1d-bbd5-4528-ba52-bcc9fb43897b") : secret "maas-api-e2e-ait-26b8d994-serving-cert" not found
Nov 03 22:48:09.717098 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.717025    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-openshift-service-ca\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.717168 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.717154    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdnvr\" (UniqueName: \"kubernetes.io/projected/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-kube-api-access-qdnvr\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.717739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.717712    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-openshift-service-ca\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.719573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.719548    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-metrics-certs\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:09.818546 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:09.818458    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdnvr\" (UniqueName: \"kubernetes.io/projected/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-kube-api-access-qdnvr\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:10.246925 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:10.246109    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:10.253222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:10.253174    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls\") pod \"maas-api-e2e-ait-26b8d994-b5977797-2q4bq\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:10.438693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:10.437059    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:11.219835 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:48:11.219803    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2122b1d_bbd5_4528_ba52_bcc9fb43897b.slice/crio-59da2a194f7171fb4141e69a45d24e80b6d260b67778f6890ff42e1ed43515ea WatchSource:0}: Error finding container 59da2a194f7171fb4141e69a45d24e80b6d260b67778f6890ff42e1ed43515ea: Status 404 returned error can't find the container with id 59da2a194f7171fb4141e69a45d24e80b6d260b67778f6890ff42e1ed43515ea
Nov 03 22:48:11.220568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:11.220545    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"]
Nov 03 22:48:11.829623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:11.829566    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq" event={"ID":"a2122b1d-bbd5-4528-ba52-bcc9fb43897b","Type":"ContainerStarted","Data":"59da2a194f7171fb4141e69a45d24e80b6d260b67778f6890ff42e1ed43515ea"}
Nov 03 22:48:12.956901 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:12.956838    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq" event={"ID":"a2122b1d-bbd5-4528-ba52-bcc9fb43897b","Type":"ContainerStarted","Data":"51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477"}
Nov 03 22:48:12.957585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:12.957560    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:13.074102 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.074030    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq" podStartSLOduration=3.30961793 podStartE2EDuration="4.074010135s" totalImagesPullingTime="764.392205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:48:13.073486044 +0000 UTC m=+1516.514023305" watchObservedRunningTime="2026-11-03 22:48:13.074010135 +0000 UTC m=+1516.514547393"
Nov 03 22:48:13.416883 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.416768    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6" podUID="fc2ae31d-4cf4-40bf-a192-f2fac423d64f" containerName="payload-processing" containerID="cri-o://3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac" gracePeriod=25
Nov 03 22:48:13.535262 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.535185    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw" podUID="bb24e377-8d98-428e-ad53-e70eb4a18e13" containerName="payload-pre-processing" containerID="cri-o://5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e" gracePeriod=25
Nov 03 22:48:13.723433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.723403    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:48:13.885818 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.885788    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:48:13.888550 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.888522    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb24e377-8d98-428e-ad53-e70eb4a18e13-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb24e377-8d98-428e-ad53-e70eb4a18e13-plugins-config-volume\") pod \"bb24e377-8d98-428e-ad53-e70eb4a18e13\" (UID: \"bb24e377-8d98-428e-ad53-e70eb4a18e13\") "
Nov 03 22:48:13.888807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.888573    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb24e377-8d98-428e-ad53-e70eb4a18e13-kube-api-access-5ptjk\" (UniqueName: \"kubernetes.io/projected/bb24e377-8d98-428e-ad53-e70eb4a18e13-kube-api-access-5ptjk\") pod \"bb24e377-8d98-428e-ad53-e70eb4a18e13\" (UID: \"bb24e377-8d98-428e-ad53-e70eb4a18e13\") "
Nov 03 22:48:13.888807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.888722    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-plugins-config-volume\") pod \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\" (UID: \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\") "
Nov 03 22:48:13.888974 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.888957    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-kube-api-access-nl64v\" (UniqueName: \"kubernetes.io/projected/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-kube-api-access-nl64v\") pod \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\" (UID: \"fc2ae31d-4cf4-40bf-a192-f2fac423d64f\") "
Nov 03 22:48:13.889221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.888949    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb24e377-8d98-428e-ad53-e70eb4a18e13-plugins-config-volume" pod "bb24e377-8d98-428e-ad53-e70eb4a18e13" (UID: "bb24e377-8d98-428e-ad53-e70eb4a18e13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:48:13.889221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.889104    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-plugins-config-volume" pod "fc2ae31d-4cf4-40bf-a192-f2fac423d64f" (UID: "fc2ae31d-4cf4-40bf-a192-f2fac423d64f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:48:13.889627 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.889607    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb24e377-8d98-428e-ad53-e70eb4a18e13-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:13.889760 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.889629    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:13.891221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.891192    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb24e377-8d98-428e-ad53-e70eb4a18e13-kube-api-access-5ptjk" pod "bb24e377-8d98-428e-ad53-e70eb4a18e13" (UID: "bb24e377-8d98-428e-ad53-e70eb4a18e13"). InnerVolumeSpecName "kube-api-access-5ptjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:48:13.891584 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.891566    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-kube-api-access-nl64v" pod "fc2ae31d-4cf4-40bf-a192-f2fac423d64f" (UID: "fc2ae31d-4cf4-40bf-a192-f2fac423d64f"). InnerVolumeSpecName "kube-api-access-nl64v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:48:13.965599 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.965551    5647 generic.go:356] "Generic (PLEG): container finished" podID="fc2ae31d-4cf4-40bf-a192-f2fac423d64f" containerID="3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac" exitCode=0
Nov 03 22:48:13.965599 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.965593    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"
Nov 03 22:48:13.966140 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.965646    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6" event={"ID":"fc2ae31d-4cf4-40bf-a192-f2fac423d64f","Type":"ContainerDied","Data":"3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac"}
Nov 03 22:48:13.966140 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.965719    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6" event={"ID":"fc2ae31d-4cf4-40bf-a192-f2fac423d64f","Type":"ContainerDied","Data":"4c359c649daea0c15dfecc9bb13b1ccae4bbf020697e62f8e3bd12f169947fdd"}
Nov 03 22:48:13.966140 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.965751    5647 scope.go:122] "RemoveContainer" containerID="3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac"
Nov 03 22:48:13.967319 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.967247    5647 generic.go:356] "Generic (PLEG): container finished" podID="bb24e377-8d98-428e-ad53-e70eb4a18e13" containerID="5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e" exitCode=0
Nov 03 22:48:13.967319 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.967305    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"
Nov 03 22:48:13.967319 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.967338    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw" event={"ID":"bb24e377-8d98-428e-ad53-e70eb4a18e13","Type":"ContainerDied","Data":"5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e"}
Nov 03 22:48:13.967660 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.967382    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw" event={"ID":"bb24e377-8d98-428e-ad53-e70eb4a18e13","Type":"ContainerDied","Data":"aff59b27996145f9f8ee5b06043105b4607b7169f1a1deff33f01855a653d0fa"}
Nov 03 22:48:13.977563 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.977541    5647 scope.go:122] "RemoveContainer" containerID="3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac"
Nov 03 22:48:13.977942 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:48:13.977919    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac\": container with ID starting with 3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac not found: ID does not exist" containerID="3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac"
Nov 03 22:48:13.978001 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.977950    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac"} err="failed to get container status \"3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac\": rpc error: code = NotFound desc = could not find container \"3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac\": container with ID starting with 3d4eb931bd1ddc53412d3d17115ca9f22ecff0112555c330f8fec04722d9c5ac not found: ID does not exist"
Nov 03 22:48:13.978001 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.977966    5647 scope.go:122] "RemoveContainer" containerID="5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e"
Nov 03 22:48:13.991189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.991164    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl64v\" (UniqueName: \"kubernetes.io/projected/fc2ae31d-4cf4-40bf-a192-f2fac423d64f-kube-api-access-nl64v\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:13.991189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:13.991191    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ptjk\" (UniqueName: \"kubernetes.io/projected/bb24e377-8d98-428e-ad53-e70eb4a18e13-kube-api-access-5ptjk\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:14.004539 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:14.004496    5647 scope.go:122] "RemoveContainer" containerID="5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e"
Nov 03 22:48:14.005015 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:48:14.004987    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e\": container with ID starting with 5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e not found: ID does not exist" containerID="5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e"
Nov 03 22:48:14.005129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:14.005027    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e"} err="failed to get container status \"5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e\": rpc error: code = NotFound desc = could not find container \"5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e\": container with ID starting with 5f894078e3d3cc362ac1528ee3915c8a2ac76d2c256ca0d12e805d4f5dd85f0e not found: ID does not exist"
Nov 03 22:48:14.114027 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:14.113992    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"]
Nov 03 22:48:14.122246 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:14.122216    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw"]
Nov 03 22:48:14.230255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:14.230153    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"]
Nov 03 22:48:14.234250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:14.234219    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6"]
Nov 03 22:48:14.698993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:14.698901    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb24e377-8d98-428e-ad53-e70eb4a18e13" path="/var/lib/kubelet/pods/bb24e377-8d98-428e-ad53-e70eb4a18e13/volumes"
Nov 03 22:48:14.699466 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:14.699443    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc2ae31d-4cf4-40bf-a192-f2fac423d64f" path="/var/lib/kubelet/pods/fc2ae31d-4cf4-40bf-a192-f2fac423d64f/volumes"
Nov 03 22:48:16.046389 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:48:16.046351    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:48:16.272625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:16.272590    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:48:18.973608 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:18.973574    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:48:19.268992 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.268894    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"]
Nov 03 22:48:19.269782 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.269761    5647 cpu_manager.go:414] "containerMap: removing container" podUID="bb24e377-8d98-428e-ad53-e70eb4a18e13" containerName="payload-pre-processing"
Nov 03 22:48:19.269782 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.269782    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb24e377-8d98-428e-ad53-e70eb4a18e13" containerName="payload-pre-processing"
Nov 03 22:48:19.269962 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.269822    5647 cpu_manager.go:414] "containerMap: removing container" podUID="fc2ae31d-4cf4-40bf-a192-f2fac423d64f" containerName="payload-processing"
Nov 03 22:48:19.269962 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.269830    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc2ae31d-4cf4-40bf-a192-f2fac423d64f" containerName="payload-processing"
Nov 03 22:48:19.270091 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.270076    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb24e377-8d98-428e-ad53-e70eb4a18e13" containerName="payload-pre-processing"
Nov 03 22:48:19.270141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.270101    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc2ae31d-4cf4-40bf-a192-f2fac423d64f" containerName="payload-processing"
Nov 03 22:48:19.273870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.273849    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.328246 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.328059    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9d7aa682-serving-cert\""
Nov 03 22:48:19.333715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.333687    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"]
Nov 03 22:48:19.425482 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.425445    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-maas-api-tls\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.425690 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.425512    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-openshift-service-ca\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.425810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.425781    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-metrics-certs\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.426031 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.426010    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qddsc\" (UniqueName: \"kubernetes.io/projected/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-kube-api-access-qddsc\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.539739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.537140    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-metrics-certs\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.539739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.537426    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qddsc\" (UniqueName: \"kubernetes.io/projected/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-kube-api-access-qddsc\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.539739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.537633    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-maas-api-tls\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.539739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.537704    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-openshift-service-ca\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.539739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.538792    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-openshift-service-ca\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.605850 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.605776    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-maas-api-tls\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.625107 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.623971    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-metrics-certs\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.651456 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.651383    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qddsc\" (UniqueName: \"kubernetes.io/projected/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-kube-api-access-qddsc\") pod \"maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:19.889394 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:19.889298    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:20.279014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:20.278984    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"]
Nov 03 22:48:20.279365 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:48:20.279147    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bee21e7_1482_44e3_8fb1_5816ce1cc2bf.slice/crio-a1e64b01d2edeb3139b864d8c1545c7d1e5bd1e171ed40c830e0770041332e32 WatchSource:0}: Error finding container a1e64b01d2edeb3139b864d8c1545c7d1e5bd1e171ed40c830e0770041332e32: Status 404 returned error can't find the container with id a1e64b01d2edeb3139b864d8c1545c7d1e5bd1e171ed40c830e0770041332e32
Nov 03 22:48:21.054361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:21.054326    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz" event={"ID":"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf","Type":"ContainerStarted","Data":"a1e64b01d2edeb3139b864d8c1545c7d1e5bd1e171ed40c830e0770041332e32"}
Nov 03 22:48:22.064472 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:22.064432    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz" event={"ID":"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf","Type":"ContainerStarted","Data":"73dde6e7dc6dd56648fd3ee5355e1018f371535311caafeb2ebb14fa1ccbeb8b"}
Nov 03 22:48:22.064914 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:22.064532    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:22.187718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:22.187646    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz" podStartSLOduration=2.352674884 podStartE2EDuration="3.187628403s" totalImagesPullingTime="834.953519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:48:22.1853528 +0000 UTC m=+1525.625890060" watchObservedRunningTime="2026-11-03 22:48:22.187628403 +0000 UTC m=+1525.628165663"
Nov 03 22:48:28.078867 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:28.078834    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:48:39.492826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.492762    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"]
Nov 03 22:48:39.509818 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.509791    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:39.554695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.528613    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"]
Nov 03 22:48:39.602099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.602048    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mc8r\" (UniqueName: \"kubernetes.io/projected/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-kube-api-access-4mc8r\") pod \"payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4\" (UID: \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\") " pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:39.602779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.602622    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-plugins-config-volume\") pod \"payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4\" (UID: \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\") " pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:39.752813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.708231    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mc8r\" (UniqueName: \"kubernetes.io/projected/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-kube-api-access-4mc8r\") pod \"payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4\" (UID: \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\") " pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:39.844938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.801612    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-plugins-config-volume\") pod \"payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4\" (UID: \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\") " pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:39.844938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.770165    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ebffae22\""
Nov 03 22:48:39.844938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.804850    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-plugins-config-volume\") pod \"payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4\" (UID: \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\") " pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:39.844938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.770845    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ebffae22-dockercfg-xhnxx\""
Nov 03 22:48:39.844938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.829330    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mc8r\" (UniqueName: \"kubernetes.io/projected/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-kube-api-access-4mc8r\") pod \"payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4\" (UID: \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\") " pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:39.883811 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:39.883766    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:40.273458 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:40.273427    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"]
Nov 03 22:48:40.273850 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:48:40.273825    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad2f6eec_aab0_4ea6_83e8_5ea05cd62afd.slice/crio-30ef31dc9664cf57e76c3ce160a85352fdbd5b3ef0b5b6ca1fd9347fc1bb56e3 WatchSource:0}: Error finding container 30ef31dc9664cf57e76c3ce160a85352fdbd5b3ef0b5b6ca1fd9347fc1bb56e3: Status 404 returned error can't find the container with id 30ef31dc9664cf57e76c3ce160a85352fdbd5b3ef0b5b6ca1fd9347fc1bb56e3
Nov 03 22:48:40.745139 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:40.745106    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4" event={"ID":"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd","Type":"ContainerStarted","Data":"f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346"}
Nov 03 22:48:40.745560 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:40.745158    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4" event={"ID":"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd","Type":"ContainerStarted","Data":"30ef31dc9664cf57e76c3ce160a85352fdbd5b3ef0b5b6ca1fd9347fc1bb56e3"}
Nov 03 22:48:40.745560 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:40.745299    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:40.867488 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:40.867436    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4" podStartSLOduration=1.867421636 podStartE2EDuration="1.867421636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:48:40.862996367 +0000 UTC m=+1544.303533627" watchObservedRunningTime="2026-11-03 22:48:40.867421636 +0000 UTC m=+1544.307958894"
Nov 03 22:48:42.070595 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:42.070559    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"]
Nov 03 22:48:42.071077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:42.070835    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy" containerID="cri-o://764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320" gracePeriod=30
Nov 03 22:48:46.139289 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:48:46.139253    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:48:47.192812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.192401    5647 patch_prober.go:29] interesting pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Nov 03 22:48:47.192812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.192485    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Nov 03 22:48:47.411519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.411493    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:48:47.558014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.557979    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-token\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-token\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.558205 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558130    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-podinfo\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.558205 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558192    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-credential-socket\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.558343 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558222    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-certs\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.558343 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558247    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9daf04a2-44f1-4659-b8ce-ad117f28527d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9daf04a2-44f1-4659-b8ce-ad117f28527d-istiod-ca-cert\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.558446 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558350    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-data\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558500    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-envoy\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558522    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-credential-socket" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558566    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-socket\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558638    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-certs" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558651    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-kube-api-access-t55x5\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-kube-api-access-t55x5\") pod \"9daf04a2-44f1-4659-b8ce-ad117f28527d\" (UID: \"9daf04a2-44f1-4659-b8ce-ad117f28527d\") "
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558773    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-data" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.558897    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-socket" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.559698    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.559722    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.559736    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.559751    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.559966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.559933    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9daf04a2-44f1-4659-b8ce-ad117f28527d-istiod-ca-cert" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:48:47.560897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.560866    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-envoy" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:48:47.561287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.561260    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-podinfo" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:48:47.561371 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.561355    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-kube-api-access-t55x5" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "kube-api-access-t55x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:48:47.562795 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.562693    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-token" pod "9daf04a2-44f1-4659-b8ce-ad117f28527d" (UID: "9daf04a2-44f1-4659-b8ce-ad117f28527d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:48:47.822720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.805171    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.822720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.805206    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t55x5\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-kube-api-access-t55x5\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.822720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.805232    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.822720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.805249    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9daf04a2-44f1-4659-b8ce-ad117f28527d-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.822720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.805264    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9daf04a2-44f1-4659-b8ce-ad117f28527d-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:48:47.903827 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.903795    5647 generic.go:356] "Generic (PLEG): container finished" podID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerID="764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320" exitCode=0
Nov 03 22:48:47.904005 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.903874    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"
Nov 03 22:48:47.904005 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.903885    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" event={"ID":"9daf04a2-44f1-4659-b8ce-ad117f28527d","Type":"ContainerDied","Data":"764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320"}
Nov 03 22:48:47.904005 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.903973    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs" event={"ID":"9daf04a2-44f1-4659-b8ce-ad117f28527d","Type":"ContainerDied","Data":"8d77a0e827c0e50c2521111d7c902d82b44e2f17b4bb06c3ab0fd4fa6e3179e5"}
Nov 03 22:48:47.904005 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.903995    5647 scope.go:122] "RemoveContainer" containerID="764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320"
Nov 03 22:48:47.916172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.916149    5647 scope.go:122] "RemoveContainer" containerID="b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84"
Nov 03 22:48:47.926398 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.926377    5647 scope.go:122] "RemoveContainer" containerID="764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320"
Nov 03 22:48:47.926697 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:48:47.926658    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320\": container with ID starting with 764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320 not found: ID does not exist" containerID="764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320"
Nov 03 22:48:47.926781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.926711    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320"} err="failed to get container status \"764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320\": rpc error: code = NotFound desc = could not find container \"764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320\": container with ID starting with 764badc95bbda0501af06689ca5aaa185e8febe621d1a982da3b5dcf4f0bd320 not found: ID does not exist"
Nov 03 22:48:47.926781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.926738    5647 scope.go:122] "RemoveContainer" containerID="b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84"
Nov 03 22:48:47.927025 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:48:47.927006    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84\": container with ID starting with b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84 not found: ID does not exist" containerID="b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84"
Nov 03 22:48:47.927067 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:47.927031    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84"} err="failed to get container status \"b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84\": rpc error: code = NotFound desc = could not find container \"b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84\": container with ID starting with b2a96ffc622346fb02a5fd80ce68d2e090a43faff282d1fdd23d74a5e4f6de84 not found: ID does not exist"
Nov 03 22:48:48.070807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:48.070770    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"]
Nov 03 22:48:48.119862 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:48.119768    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs"]
Nov 03 22:48:48.758695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:48.753358    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" path="/var/lib/kubelet/pods/9daf04a2-44f1-4659-b8ce-ad117f28527d/volumes"
Nov 03 22:48:50.939585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:50.939545    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"]
Nov 03 22:48:50.940573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:50.940551    5647 cpu_manager.go:414] "containerMap: removing container" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy"
Nov 03 22:48:50.940573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:50.940573    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy"
Nov 03 22:48:50.940781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:50.940652    5647 cpu_manager.go:414] "containerMap: removing container" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy"
Nov 03 22:48:50.940781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:50.940661    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy"
Nov 03 22:48:50.940961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:50.940922    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy"
Nov 03 22:48:50.940961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:50.940946    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="9daf04a2-44f1-4659-b8ce-ad117f28527d" containerName="istio-proxy"
Nov 03 22:48:50.946099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:50.946077    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014050    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014140    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014256    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014374    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp48b\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-kube-api-access-zp48b\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014452    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014475    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014576    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014629    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.014701    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.078752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.016064    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a9ab23c4-openshift-default-dockercfg-rxq5w\""
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.116438    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.116567    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.116715    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp48b\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-kube-api-access-zp48b\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.116797    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.116821    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.116943    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.116999    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.117057    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.117694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.117202    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.118292 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.117732    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.179691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.132171    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.179691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.135064    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.179691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.135615    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.179691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.136218    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.179691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.169649    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"]
Nov 03 22:48:51.240080 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.201909    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp48b\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-kube-api-access-zp48b\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.240080 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.206326    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.270572 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.241441    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.286078 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.284735    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.295717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.292028    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:51.754568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:51.754480    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:48:52.003567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:52.003535    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"]
Nov 03 22:48:52.004470 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:48:52.004445    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83a20cc8_b64c_42d3_8d53_3a2a04f49735.slice/crio-11a7314107f0fdf1d1599fe9b6dcd4c3d2fcb8d2cf085006c257da49daf5607e WatchSource:0}: Error finding container 11a7314107f0fdf1d1599fe9b6dcd4c3d2fcb8d2cf085006c257da49daf5607e: Status 404 returned error can't find the container with id 11a7314107f0fdf1d1599fe9b6dcd4c3d2fcb8d2cf085006c257da49daf5607e
Nov 03 22:48:52.007332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:52.007296    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:48:52.007424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:52.007364    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:48:52.007424 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:52.007394    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:48:52.081198 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:52.081170    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" event={"ID":"83a20cc8-b64c-42d3-8d53-3a2a04f49735","Type":"ContainerStarted","Data":"11a7314107f0fdf1d1599fe9b6dcd4c3d2fcb8d2cf085006c257da49daf5607e"}
Nov 03 22:48:52.720905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:52.720862    5647 patch_prober.go:29] interesting pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:48:52.721087 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:52.720937    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:48:53.091479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:53.091396    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" event={"ID":"83a20cc8-b64c-42d3-8d53-3a2a04f49735","Type":"ContainerStarted","Data":"dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c"}
Nov 03 22:48:53.266141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:53.266075    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" podStartSLOduration=3.266053761 podStartE2EDuration="3.266053761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:48:53.260660925 +0000 UTC m=+1556.701198185" watchObservedRunningTime="2026-11-03 22:48:53.266053761 +0000 UTC m=+1556.706591021"
Nov 03 22:48:53.292991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:53.292958    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:53.299942 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:53.299914    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:54.188468 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:54.188417    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:54.191871 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:54.191843    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:48:56.272021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:56.271978    5647 generic.go:356] "Generic (PLEG): container finished" podID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerID="8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822" exitCode=0
Nov 03 22:48:56.273961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:56.273930    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" event={"ID":"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa","Type":"ContainerDied","Data":"8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822"}
Nov 03 22:48:56.274847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:56.274826    5647 scope.go:122] "RemoveContainer" containerID="8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822"
Nov 03 22:48:56.286425 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:56.286378    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:48:56.288366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:56.288333    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:48:56.288581 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:56.288561    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:48:57.304151 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:57.304111    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" event={"ID":"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa","Type":"ContainerStarted","Data":"2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718"}
Nov 03 22:48:57.898148 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:57.898111    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:48:57.898407 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:57.898378    5647 patch_prober.go:29] interesting pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:48:57.898735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:57.898631    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:48:58.898275 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:58.898238    5647 patch_prober.go:29] interesting pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:48:58.898761 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:58.898292    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:48:59.897876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:59.897840    5647 patch_prober.go:29] interesting pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:48:59.898184 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:48:59.898131    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:49:00.610353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.610312    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"]
Nov 03 22:49:00.629706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.627368    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:00.674287 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.674240    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"]
Nov 03 22:49:00.778722 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.776204    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97183b8c-48d7-4950-9a67-327c011f0af6-plugins-config-volume\") pod \"payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs\" (UID: \"97183b8c-48d7-4950-9a67-327c011f0af6\") " pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:00.778722 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.776558    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx8h7\" (UniqueName: \"kubernetes.io/projected/97183b8c-48d7-4950-9a67-327c011f0af6-kube-api-access-cx8h7\") pod \"payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs\" (UID: \"97183b8c-48d7-4950-9a67-327c011f0af6\") " pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:00.778722 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.777238    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-a9ab23c4-dockercfg-nrt6w\""
Nov 03 22:49:00.778722 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.777323    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a9ab23c4\""
Nov 03 22:49:00.951765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.926215    5647 patch_prober.go:29] interesting pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:49:00.952070 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.952027    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:49:00.952228 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.935876    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97183b8c-48d7-4950-9a67-327c011f0af6-plugins-config-volume\") pod \"payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs\" (UID: \"97183b8c-48d7-4950-9a67-327c011f0af6\") " pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:00.952361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.934967    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97183b8c-48d7-4950-9a67-327c011f0af6-plugins-config-volume\") pod \"payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs\" (UID: \"97183b8c-48d7-4950-9a67-327c011f0af6\") " pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:00.952999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:00.952970    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx8h7\" (UniqueName: \"kubernetes.io/projected/97183b8c-48d7-4950-9a67-327c011f0af6-kube-api-access-cx8h7\") pod \"payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs\" (UID: \"97183b8c-48d7-4950-9a67-327c011f0af6\") " pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:01.260189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.211824    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"]
Nov 03 22:49:01.310353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.306054    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"]
Nov 03 22:49:01.316948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.316916    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:01.345112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.331848    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx8h7\" (UniqueName: \"kubernetes.io/projected/97183b8c-48d7-4950-9a67-327c011f0af6-kube-api-access-cx8h7\") pod \"payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs\" (UID: \"97183b8c-48d7-4950-9a67-327c011f0af6\") " pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:01.398702 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.395306    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:01.453429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.442739    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:49:01.461821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.456104    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fb2147c-d3fa-491e-be7b-e4be0b669502-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb\" (UID: \"1fb2147c-d3fa-491e-be7b-e4be0b669502\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:01.461821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.456193    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gmhj\" (UniqueName: \"kubernetes.io/projected/1fb2147c-d3fa-491e-be7b-e4be0b669502-kube-api-access-4gmhj\") pod \"payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb\" (UID: \"1fb2147c-d3fa-491e-be7b-e4be0b669502\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:01.557985 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.557903    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fb2147c-d3fa-491e-be7b-e4be0b669502-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb\" (UID: \"1fb2147c-d3fa-491e-be7b-e4be0b669502\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:01.558153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.557996    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gmhj\" (UniqueName: \"kubernetes.io/projected/1fb2147c-d3fa-491e-be7b-e4be0b669502-kube-api-access-4gmhj\") pod \"payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb\" (UID: \"1fb2147c-d3fa-491e-be7b-e4be0b669502\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:01.559274 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.559244    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fb2147c-d3fa-491e-be7b-e4be0b669502-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb\" (UID: \"1fb2147c-d3fa-491e-be7b-e4be0b669502\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:01.620550 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.620516    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gmhj\" (UniqueName: \"kubernetes.io/projected/1fb2147c-d3fa-491e-be7b-e4be0b669502-kube-api-access-4gmhj\") pod \"payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb\" (UID: \"1fb2147c-d3fa-491e-be7b-e4be0b669502\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:01.836024 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.835149    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:01.918813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.918765    5647 patch_prober.go:29] interesting pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:49:01.918967 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.918835    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:49:01.921769 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:01.921742    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"]
Nov 03 22:49:02.344166 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.344137    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"]
Nov 03 22:49:02.345015 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:49:02.344979    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fb2147c_d3fa_491e_be7b_e4be0b669502.slice/crio-9895d6cc6287f0418b59fa455d29ccf9f30678825a7afc33eaa6087d97a7b5a0 WatchSource:0}: Error finding container 9895d6cc6287f0418b59fa455d29ccf9f30678825a7afc33eaa6087d97a7b5a0: Status 404 returned error can't find the container with id 9895d6cc6287f0418b59fa455d29ccf9f30678825a7afc33eaa6087d97a7b5a0
Nov 03 22:49:02.665332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.665229    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs" event={"ID":"97183b8c-48d7-4950-9a67-327c011f0af6","Type":"ContainerStarted","Data":"167e23dc99d7aa330b8ddfb840f016bbf60af5c413f58dc7ea5115367c7e7f9a"}
Nov 03 22:49:02.665332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.665299    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:02.665332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.665324    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs" event={"ID":"97183b8c-48d7-4950-9a67-327c011f0af6","Type":"ContainerStarted","Data":"94543019c0b3b5f3b62f3a8aba015e6abcc707af2d51bcf8571d04db5fcf7b8e"}
Nov 03 22:49:02.666931 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.666904    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb" event={"ID":"1fb2147c-d3fa-491e-be7b-e4be0b669502","Type":"ContainerStarted","Data":"116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35"}
Nov 03 22:49:02.667035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.666954    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb" event={"ID":"1fb2147c-d3fa-491e-be7b-e4be0b669502","Type":"ContainerStarted","Data":"9895d6cc6287f0418b59fa455d29ccf9f30678825a7afc33eaa6087d97a7b5a0"}
Nov 03 22:49:02.667035 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.666986    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:02.799767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.799715    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs" podStartSLOduration=2.799702085 podStartE2EDuration="2.799702085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:49:02.796004496 +0000 UTC m=+1566.236541756" watchObservedRunningTime="2026-11-03 22:49:02.799702085 +0000 UTC m=+1566.240239344"
Nov 03 22:49:02.909805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.909620    5647 patch_prober.go:29] interesting pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Nov 03 22:49:02.909805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.909702    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Nov 03 22:49:02.944522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:02.937197    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb" podStartSLOduration=2.9371738130000002 podStartE2EDuration="2.937173813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:49:02.916957623 +0000 UTC m=+1566.357494883" watchObservedRunningTime="2026-11-03 22:49:02.937173813 +0000 UTC m=+1566.377711073"
Nov 03 22:49:03.903271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:03.903244    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:49:03.903727 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:03.903643    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:49:03.904457 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:03.904438    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:49:13.677752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:13.677719    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:49:13.678305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:13.677788    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:49:16.045905 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:49:16.045865    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:49:21.230514 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:21.229250    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:49:26.493434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:26.493389    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"]
Nov 03 22:49:26.603235 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:26.603199    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"]
Nov 03 22:49:31.496558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.496520    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch" podUID="b6e24077-c448-45b0-bb3c-be6f41cedc5f" containerName="payload-processing" containerID="cri-o://2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15" gracePeriod=25
Nov 03 22:49:31.604034 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.603970    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9" podUID="40dea850-f578-48f8-b11c-5a47e40699d3" containerName="payload-pre-processing" containerID="cri-o://2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49" gracePeriod=25
Nov 03 22:49:31.816728 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.816700    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:49:31.908177 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.908150    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6e24077-c448-45b0-bb3c-be6f41cedc5f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6e24077-c448-45b0-bb3c-be6f41cedc5f-plugins-config-volume\") pod \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\" (UID: \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\") "
Nov 03 22:49:31.908317 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.908244    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6e24077-c448-45b0-bb3c-be6f41cedc5f-kube-api-access-8xl2t\" (UniqueName: \"kubernetes.io/projected/b6e24077-c448-45b0-bb3c-be6f41cedc5f-kube-api-access-8xl2t\") pod \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\" (UID: \"b6e24077-c448-45b0-bb3c-be6f41cedc5f\") "
Nov 03 22:49:31.908565 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.908544    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e24077-c448-45b0-bb3c-be6f41cedc5f-plugins-config-volume" pod "b6e24077-c448-45b0-bb3c-be6f41cedc5f" (UID: "b6e24077-c448-45b0-bb3c-be6f41cedc5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:49:31.909054 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.909019    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6e24077-c448-45b0-bb3c-be6f41cedc5f-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:49:31.910481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.910460    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e24077-c448-45b0-bb3c-be6f41cedc5f-kube-api-access-8xl2t" pod "b6e24077-c448-45b0-bb3c-be6f41cedc5f" (UID: "b6e24077-c448-45b0-bb3c-be6f41cedc5f"). InnerVolumeSpecName "kube-api-access-8xl2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:49:31.944401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:31.944365    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:49:32.010433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.010394    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40dea850-f578-48f8-b11c-5a47e40699d3-kube-api-access-nz5pg\" (UniqueName: \"kubernetes.io/projected/40dea850-f578-48f8-b11c-5a47e40699d3-kube-api-access-nz5pg\") pod \"40dea850-f578-48f8-b11c-5a47e40699d3\" (UID: \"40dea850-f578-48f8-b11c-5a47e40699d3\") "
Nov 03 22:49:32.010612 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.010517    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40dea850-f578-48f8-b11c-5a47e40699d3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40dea850-f578-48f8-b11c-5a47e40699d3-plugins-config-volume\") pod \"40dea850-f578-48f8-b11c-5a47e40699d3\" (UID: \"40dea850-f578-48f8-b11c-5a47e40699d3\") "
Nov 03 22:49:32.011009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.010976    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40dea850-f578-48f8-b11c-5a47e40699d3-plugins-config-volume" pod "40dea850-f578-48f8-b11c-5a47e40699d3" (UID: "40dea850-f578-48f8-b11c-5a47e40699d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:49:32.011370 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.011344    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xl2t\" (UniqueName: \"kubernetes.io/projected/b6e24077-c448-45b0-bb3c-be6f41cedc5f-kube-api-access-8xl2t\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:49:32.011465 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.011377    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40dea850-f578-48f8-b11c-5a47e40699d3-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:49:32.013453 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.013417    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40dea850-f578-48f8-b11c-5a47e40699d3-kube-api-access-nz5pg" pod "40dea850-f578-48f8-b11c-5a47e40699d3" (UID: "40dea850-f578-48f8-b11c-5a47e40699d3"). InnerVolumeSpecName "kube-api-access-nz5pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:49:32.114302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.114181    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz5pg\" (UniqueName: \"kubernetes.io/projected/40dea850-f578-48f8-b11c-5a47e40699d3-kube-api-access-nz5pg\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:49:32.395578 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.395486    5647 generic.go:356] "Generic (PLEG): container finished" podID="b6e24077-c448-45b0-bb3c-be6f41cedc5f" containerID="2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15" exitCode=0
Nov 03 22:49:32.396072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.396056    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"
Nov 03 22:49:32.396805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.396771    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch" event={"ID":"b6e24077-c448-45b0-bb3c-be6f41cedc5f","Type":"ContainerDied","Data":"2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15"}
Nov 03 22:49:32.396912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.396843    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch" event={"ID":"b6e24077-c448-45b0-bb3c-be6f41cedc5f","Type":"ContainerDied","Data":"37a9f8a01db0724fe6ef96d0fa9f5beb0e270aaaafc2750c78463cee2e675392"}
Nov 03 22:49:32.396912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.396879    5647 scope.go:122] "RemoveContainer" containerID="2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15"
Nov 03 22:49:32.398914 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.398890    5647 generic.go:356] "Generic (PLEG): container finished" podID="40dea850-f578-48f8-b11c-5a47e40699d3" containerID="2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49" exitCode=0
Nov 03 22:49:32.399016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.398943    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"
Nov 03 22:49:32.399016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.398989    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9" event={"ID":"40dea850-f578-48f8-b11c-5a47e40699d3","Type":"ContainerDied","Data":"2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49"}
Nov 03 22:49:32.399111 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.399031    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9" event={"ID":"40dea850-f578-48f8-b11c-5a47e40699d3","Type":"ContainerDied","Data":"af76fb4da2c2cba02a9febe6dbd251440347aed6378b8d5b73c4b50b05f56877"}
Nov 03 22:49:32.412122 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.412097    5647 scope.go:122] "RemoveContainer" containerID="2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15"
Nov 03 22:49:32.412987 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:49:32.412967    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15\": container with ID starting with 2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15 not found: ID does not exist" containerID="2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15"
Nov 03 22:49:32.413054 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.412994    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15"} err="failed to get container status \"2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15\": rpc error: code = NotFound desc = could not find container \"2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15\": container with ID starting with 2eb8824f34d0fc43de8db7b266e6b9a1e37f192558b6c0052c4ed685850fdb15 not found: ID does not exist"
Nov 03 22:49:32.413054 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.413010    5647 scope.go:122] "RemoveContainer" containerID="2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49"
Nov 03 22:49:32.424109 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.424090    5647 scope.go:122] "RemoveContainer" containerID="2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49"
Nov 03 22:49:32.424407 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:49:32.424388    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49\": container with ID starting with 2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49 not found: ID does not exist" containerID="2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49"
Nov 03 22:49:32.424464 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.424415    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49"} err="failed to get container status \"2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49\": rpc error: code = NotFound desc = could not find container \"2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49\": container with ID starting with 2ba6d073e779aae10dc65ff50b8a038e981ae93d62f6a0cad13534378ba4cb49 not found: ID does not exist"
Nov 03 22:49:32.642527 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.642475    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"]
Nov 03 22:49:32.654066 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.653990    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch"]
Nov 03 22:49:32.699965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.699927    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6e24077-c448-45b0-bb3c-be6f41cedc5f" path="/var/lib/kubelet/pods/b6e24077-c448-45b0-bb3c-be6f41cedc5f/volumes"
Nov 03 22:49:32.703072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.703045    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"]
Nov 03 22:49:32.709061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.709024    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9"]
Nov 03 22:49:32.935072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:32.934976    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9" podUID="40dea850-f578-48f8-b11c-5a47e40699d3" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.97:50052\" within 1s: context deadline exceeded"
Nov 03 22:49:32.935072 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:49:32.935066    5647 logging.go:55] [core] [Channel #2564 SubChannel #2565]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.97:50052", ServerName: "10.134.0.97:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.97:50052: operation was canceled"
Nov 03 22:49:33.411516 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:33.411488    5647 generic.go:356] "Generic (PLEG): container finished" podID="30be5938-7f33-495a-8537-a012b282e83a" containerID="353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9" exitCode=0
Nov 03 22:49:33.411706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:33.411556    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" event={"ID":"30be5938-7f33-495a-8537-a012b282e83a","Type":"ContainerDied","Data":"353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9"}
Nov 03 22:49:33.412078 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:33.412054    5647 scope.go:122] "RemoveContainer" containerID="353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9"
Nov 03 22:49:33.413748 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:33.413722    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:49:33.413828 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:33.413792    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:49:33.413828 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:33.413822    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:49:34.422729 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:34.422690    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" event={"ID":"30be5938-7f33-495a-8537-a012b282e83a","Type":"ContainerStarted","Data":"0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7"}
Nov 03 22:49:34.511366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:34.511335    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:49:34.512633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:34.512600    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:49:34.512775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:34.512663    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:49:34.706288 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:34.706258    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40dea850-f578-48f8-b11c-5a47e40699d3" path="/var/lib/kubelet/pods/40dea850-f578-48f8-b11c-5a47e40699d3/volumes"
Nov 03 22:49:35.511641 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:35.511602    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:49:35.512082 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:35.511680    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:49:36.511897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:36.511860    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:49:36.512345 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:36.511935    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:49:37.436606 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:37.436576    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:49:37.511515 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:37.511478    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:49:37.511739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:37.511535    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:49:37.727331 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:37.727297    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"]
Nov 03 22:49:37.855540 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:37.855487    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"]
Nov 03 22:49:38.587690 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:38.584538    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:49:38.587690 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:38.584607    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:49:39.517216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:39.517168    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 03 22:49:39.519866 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:39.519818    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 03 22:49:40.515923 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:40.515892    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:49:40.517118 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:40.517096    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 22:49:42.728163 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:42.728121    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp" podUID="a5ee82ba-6c08-4c3d-9792-a8654e8c69b8" containerName="payload-processing" containerID="cri-o://1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7" gracePeriod=25
Nov 03 22:49:42.855925 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:42.855885    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b" podUID="1f9bd96a-a1b0-495b-aada-b8e6595686c4" containerName="payload-pre-processing" containerID="cri-o://30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11" gracePeriod=25
Nov 03 22:49:43.051621 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.051592    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:49:43.113414 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.113339    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-plugins-config-volume\") pod \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\" (UID: \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\") "
Nov 03 22:49:43.113414 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.113386    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-kube-api-access-mvdsn\" (UniqueName: \"kubernetes.io/projected/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-kube-api-access-mvdsn\") pod \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\" (UID: \"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8\") "
Nov 03 22:49:43.113897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.113805    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-plugins-config-volume" pod "a5ee82ba-6c08-4c3d-9792-a8654e8c69b8" (UID: "a5ee82ba-6c08-4c3d-9792-a8654e8c69b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:49:43.114219 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.114177    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:49:43.116327 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.116300    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-kube-api-access-mvdsn" pod "a5ee82ba-6c08-4c3d-9792-a8654e8c69b8" (UID: "a5ee82ba-6c08-4c3d-9792-a8654e8c69b8"). InnerVolumeSpecName "kube-api-access-mvdsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:49:43.166159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.166133    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:49:43.215289 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.215250    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f9bd96a-a1b0-495b-aada-b8e6595686c4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9bd96a-a1b0-495b-aada-b8e6595686c4-plugins-config-volume\") pod \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\" (UID: \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\") "
Nov 03 22:49:43.215467 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.215318    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f9bd96a-a1b0-495b-aada-b8e6595686c4-kube-api-access-grfbx\" (UniqueName: \"kubernetes.io/projected/1f9bd96a-a1b0-495b-aada-b8e6595686c4-kube-api-access-grfbx\") pod \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\" (UID: \"1f9bd96a-a1b0-495b-aada-b8e6595686c4\") "
Nov 03 22:49:43.215738 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.215710    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f9bd96a-a1b0-495b-aada-b8e6595686c4-plugins-config-volume" pod "1f9bd96a-a1b0-495b-aada-b8e6595686c4" (UID: "1f9bd96a-a1b0-495b-aada-b8e6595686c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:49:43.216029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.215978    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9bd96a-a1b0-495b-aada-b8e6595686c4-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:49:43.216029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.215998    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvdsn\" (UniqueName: \"kubernetes.io/projected/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8-kube-api-access-mvdsn\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:49:43.218206 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.218180    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9bd96a-a1b0-495b-aada-b8e6595686c4-kube-api-access-grfbx" pod "1f9bd96a-a1b0-495b-aada-b8e6595686c4" (UID: "1f9bd96a-a1b0-495b-aada-b8e6595686c4"). InnerVolumeSpecName "kube-api-access-grfbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:49:43.318298 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.318211    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grfbx\" (UniqueName: \"kubernetes.io/projected/1f9bd96a-a1b0-495b-aada-b8e6595686c4-kube-api-access-grfbx\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:49:43.774797 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.774764    5647 generic.go:356] "Generic (PLEG): container finished" podID="a5ee82ba-6c08-4c3d-9792-a8654e8c69b8" containerID="1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7" exitCode=0
Nov 03 22:49:43.775201 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.774813    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"
Nov 03 22:49:43.775201 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.774852    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp" event={"ID":"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8","Type":"ContainerDied","Data":"1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7"}
Nov 03 22:49:43.775201 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.774911    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp" event={"ID":"a5ee82ba-6c08-4c3d-9792-a8654e8c69b8","Type":"ContainerDied","Data":"d52ee76b657c96e99f0983e92bac70bc6c0ff5eb369ab5537ce0dee7b021e677"}
Nov 03 22:49:43.775201 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.774935    5647 scope.go:122] "RemoveContainer" containerID="1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7"
Nov 03 22:49:43.776495 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.776467    5647 generic.go:356] "Generic (PLEG): container finished" podID="1f9bd96a-a1b0-495b-aada-b8e6595686c4" containerID="30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11" exitCode=0
Nov 03 22:49:43.776786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.776517    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b" event={"ID":"1f9bd96a-a1b0-495b-aada-b8e6595686c4","Type":"ContainerDied","Data":"30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11"}
Nov 03 22:49:43.776786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.776555    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b" event={"ID":"1f9bd96a-a1b0-495b-aada-b8e6595686c4","Type":"ContainerDied","Data":"f3b1d67b53e58a3c5d8f6a2733b8817ffbb9e873ec4184f5a6460eab346e8784"}
Nov 03 22:49:43.776786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.776520    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"
Nov 03 22:49:43.786569 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.786554    5647 scope.go:122] "RemoveContainer" containerID="1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7"
Nov 03 22:49:43.786884 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:49:43.786864    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7\": container with ID starting with 1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7 not found: ID does not exist" containerID="1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7"
Nov 03 22:49:43.786967 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.786890    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7"} err="failed to get container status \"1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7\": rpc error: code = NotFound desc = could not find container \"1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7\": container with ID starting with 1ac1b1a54a5be38dba909915d757bbb23b28c1f79cea7083cfd1391e1277f5d7 not found: ID does not exist"
Nov 03 22:49:43.786967 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.786907    5647 scope.go:122] "RemoveContainer" containerID="30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11"
Nov 03 22:49:43.796642 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.796626    5647 scope.go:122] "RemoveContainer" containerID="30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11"
Nov 03 22:49:43.796989 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:49:43.796963    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11\": container with ID starting with 30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11 not found: ID does not exist" containerID="30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11"
Nov 03 22:49:43.797080 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.796998    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11"} err="failed to get container status \"30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11\": rpc error: code = NotFound desc = could not find container \"30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11\": container with ID starting with 30df146131116d3f7a65b40793fc3a8565abc8505fa9262592d15b9ab45b3b11 not found: ID does not exist"
Nov 03 22:49:43.940017 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.939970    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"]
Nov 03 22:49:43.946296 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:43.946262    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b"]
Nov 03 22:49:44.063493 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:44.063394    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"]
Nov 03 22:49:44.067494 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:44.067457    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp"]
Nov 03 22:49:44.771697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:44.767281    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f9bd96a-a1b0-495b-aada-b8e6595686c4" path="/var/lib/kubelet/pods/1f9bd96a-a1b0-495b-aada-b8e6595686c4/volumes"
Nov 03 22:49:44.771697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:44.768191    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5ee82ba-6c08-4c3d-9792-a8654e8c69b8" path="/var/lib/kubelet/pods/a5ee82ba-6c08-4c3d-9792-a8654e8c69b8/volumes"
Nov 03 22:49:46.049717 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:49:46.044582    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:49:56.361838 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.361799    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"]
Nov 03 22:49:56.362852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.362830    5647 cpu_manager.go:414] "containerMap: removing container" podUID="1f9bd96a-a1b0-495b-aada-b8e6595686c4" containerName="payload-pre-processing"
Nov 03 22:49:56.362852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.362852    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f9bd96a-a1b0-495b-aada-b8e6595686c4" containerName="payload-pre-processing"
Nov 03 22:49:56.363028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.362883    5647 cpu_manager.go:414] "containerMap: removing container" podUID="40dea850-f578-48f8-b11c-5a47e40699d3" containerName="payload-pre-processing"
Nov 03 22:49:56.363028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.362892    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40dea850-f578-48f8-b11c-5a47e40699d3" containerName="payload-pre-processing"
Nov 03 22:49:56.363028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.362931    5647 cpu_manager.go:414] "containerMap: removing container" podUID="b6e24077-c448-45b0-bb3c-be6f41cedc5f" containerName="payload-processing"
Nov 03 22:49:56.363028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.362938    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6e24077-c448-45b0-bb3c-be6f41cedc5f" containerName="payload-processing"
Nov 03 22:49:56.363028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.362995    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a5ee82ba-6c08-4c3d-9792-a8654e8c69b8" containerName="payload-processing"
Nov 03 22:49:56.363028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.363002    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5ee82ba-6c08-4c3d-9792-a8654e8c69b8" containerName="payload-processing"
Nov 03 22:49:56.363316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.363160    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6e24077-c448-45b0-bb3c-be6f41cedc5f" containerName="payload-processing"
Nov 03 22:49:56.363316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.363183    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5ee82ba-6c08-4c3d-9792-a8654e8c69b8" containerName="payload-processing"
Nov 03 22:49:56.363316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.363196    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f9bd96a-a1b0-495b-aada-b8e6595686c4" containerName="payload-pre-processing"
Nov 03 22:49:56.363316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.363208    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="40dea850-f578-48f8-b11c-5a47e40699d3" containerName="payload-pre-processing"
Nov 03 22:49:56.367552 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.367530    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.423567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.423528    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-openshift-service-ca\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.423787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.423594    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-metrics-certs\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.423872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.423777    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45dxc\" (UniqueName: \"kubernetes.io/projected/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-kube-api-access-45dxc\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.423956 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.423940    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.424113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.424091    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"]
Nov 03 22:49:56.468354 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.468319    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a9ab23c4-serving-cert\""
Nov 03 22:49:56.526060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.526028    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.526243 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:49:56.526181    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-serving-cert: secret "maas-api-e2e-mt-a9ab23c4-serving-cert" not found
Nov 03 22:49:56.526314 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:49:56.526253    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls podName:b5629a4d-425c-4cc1-bbcc-6fed7714c29d nodeName:}" failed. No retries permitted until 2026-11-03 22:49:57.026234019 +0000 UTC m=+1620.466771259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls") pod "maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj" (UID: "b5629a4d-425c-4cc1-bbcc-6fed7714c29d") : secret "maas-api-e2e-mt-a9ab23c4-serving-cert" not found
Nov 03 22:49:56.526384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.526344    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-openshift-service-ca\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.526384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.526374    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-metrics-certs\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.526500 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.526427    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45dxc\" (UniqueName: \"kubernetes.io/projected/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-kube-api-access-45dxc\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.527050 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.527027    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-openshift-service-ca\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.529101 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.529077    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-metrics-certs\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:56.587891 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:56.587861    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45dxc\" (UniqueName: \"kubernetes.io/projected/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-kube-api-access-45dxc\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:57.063205 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:57.063157    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:57.109625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:57.109592    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:57.282893 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:57.282513    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:57.701751 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:57.691533    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"]
Nov 03 22:49:58.016315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:58.016281    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj" event={"ID":"b5629a4d-425c-4cc1-bbcc-6fed7714c29d","Type":"ContainerStarted","Data":"2ca7e70bc8cf07aa00d0136078cc280356e05e6312d58d10413b421178a0b54d"}
Nov 03 22:49:59.027757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:59.027717    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj" event={"ID":"b5629a4d-425c-4cc1-bbcc-6fed7714c29d","Type":"ContainerStarted","Data":"0e10750a6e426a46111328eec61332841e3342e89767da48af95c6e686002f4f"}
Nov 03 22:49:59.028175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:59.027789    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:49:59.145450 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:49:59.145386    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj" podStartSLOduration=2.262868343 podStartE2EDuration="3.145372524s" totalImagesPullingTime="882.504181ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:49:59.143568159 +0000 UTC m=+1622.584105418" watchObservedRunningTime="2026-11-03 22:49:59.145372524 +0000 UTC m=+1622.585909816"
Nov 03 22:50:05.075899 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:05.075116    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:50:06.431314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:06.431277    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:50:07.688735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:07.688699    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"]
Nov 03 22:50:07.709568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:07.709537    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:07.734142 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:07.734096    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"]
Nov 03 22:50:07.923695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:07.886881    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-4a1480\""
Nov 03 22:50:08.020703 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:07.909893    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-4a1480-dockercfg-tnrfr\""
Nov 03 22:50:08.020703 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:07.911933    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szt4v\" (UniqueName: \"kubernetes.io/projected/61084784-f3d5-4c15-83ad-447cb2965488-kube-api-access-szt4v\") pod \"payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn\" (UID: \"61084784-f3d5-4c15-83ad-447cb2965488\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:08.020970 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:07.986385    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61084784-f3d5-4c15-83ad-447cb2965488-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn\" (UID: \"61084784-f3d5-4c15-83ad-447cb2965488\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:08.150057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.150028    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"]
Nov 03 22:50:08.201629 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.201574    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61084784-f3d5-4c15-83ad-447cb2965488-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn\" (UID: \"61084784-f3d5-4c15-83ad-447cb2965488\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:08.202175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.202153    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szt4v\" (UniqueName: \"kubernetes.io/projected/61084784-f3d5-4c15-83ad-447cb2965488-kube-api-access-szt4v\") pod \"payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn\" (UID: \"61084784-f3d5-4c15-83ad-447cb2965488\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:08.203192 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.203160    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61084784-f3d5-4c15-83ad-447cb2965488-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn\" (UID: \"61084784-f3d5-4c15-83ad-447cb2965488\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:08.210543 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.210515    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:08.215523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.215455    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"]
Nov 03 22:50:08.262683 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.262629    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szt4v\" (UniqueName: \"kubernetes.io/projected/61084784-f3d5-4c15-83ad-447cb2965488-kube-api-access-szt4v\") pod \"payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn\" (UID: \"61084784-f3d5-4c15-83ad-447cb2965488\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:08.303946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.303866    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9mkg\" (UniqueName: \"kubernetes.io/projected/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-kube-api-access-t9mkg\") pod \"payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm\" (UID: \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:08.304131 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.304107    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm\" (UID: \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:08.405638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.405601    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9mkg\" (UniqueName: \"kubernetes.io/projected/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-kube-api-access-t9mkg\") pod \"payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm\" (UID: \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:08.405876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.405859    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm\" (UID: \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:08.406558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.406541    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm\" (UID: \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:08.466154 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.466120    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9mkg\" (UniqueName: \"kubernetes.io/projected/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-kube-api-access-t9mkg\") pod \"payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm\" (UID: \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:08.483212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.483187    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:08.529816 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.529782    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:08.881020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.880935    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"]
Nov 03 22:50:08.921594 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:50:08.921563    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61084784_f3d5_4c15_83ad_447cb2965488.slice/crio-99726caf57086158ab372d35d6a23c58f7f8508f0eb14ddd281dacdc4350a655 WatchSource:0}: Error finding container 99726caf57086158ab372d35d6a23c58f7f8508f0eb14ddd281dacdc4350a655: Status 404 returned error can't find the container with id 99726caf57086158ab372d35d6a23c58f7f8508f0eb14ddd281dacdc4350a655
Nov 03 22:50:08.932583 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:08.932559    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"]
Nov 03 22:50:08.936142 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:50:08.936118    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe35d9d5_26cd_4f57_9965_1be3714dbdd4.slice/crio-5e2d79bcbc0723b7169389c829030e513f649f17c2ba1de448a7a3a4bfee94b2 WatchSource:0}: Error finding container 5e2d79bcbc0723b7169389c829030e513f649f17c2ba1de448a7a3a4bfee94b2: Status 404 returned error can't find the container with id 5e2d79bcbc0723b7169389c829030e513f649f17c2ba1de448a7a3a4bfee94b2
Nov 03 22:50:09.225736 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:09.225698    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn" event={"ID":"61084784-f3d5-4c15-83ad-447cb2965488","Type":"ContainerStarted","Data":"0b34d730505bbdd8120330c809021daae0ec11d257047690631c87ac72f2a858"}
Nov 03 22:50:09.225937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:09.225772    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:09.225937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:09.225794    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn" event={"ID":"61084784-f3d5-4c15-83ad-447cb2965488","Type":"ContainerStarted","Data":"99726caf57086158ab372d35d6a23c58f7f8508f0eb14ddd281dacdc4350a655"}
Nov 03 22:50:09.227404 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:09.227381    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm" event={"ID":"fe35d9d5-26cd-4f57-9965-1be3714dbdd4","Type":"ContainerStarted","Data":"58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a"}
Nov 03 22:50:09.227504 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:09.227421    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm" event={"ID":"fe35d9d5-26cd-4f57-9965-1be3714dbdd4","Type":"ContainerStarted","Data":"5e2d79bcbc0723b7169389c829030e513f649f17c2ba1de448a7a3a4bfee94b2"}
Nov 03 22:50:09.227504 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:09.227463    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:09.340841 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:09.340784    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn" podStartSLOduration=2.340769474 podStartE2EDuration="2.340769474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:50:09.33999623 +0000 UTC m=+1632.780533490" watchObservedRunningTime="2026-11-03 22:50:09.340769474 +0000 UTC m=+1632.781306785"
Nov 03 22:50:09.456964 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:09.456908    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm" podStartSLOduration=2.456893515 podStartE2EDuration="2.456893515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:50:09.455663175 +0000 UTC m=+1632.896200439" watchObservedRunningTime="2026-11-03 22:50:09.456893515 +0000 UTC m=+1632.897430777"
Nov 03 22:50:15.046978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:15.046920    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"]
Nov 03 22:50:15.096115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:15.096082    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c277faf-9mtv9"]
Nov 03 22:50:15.836022 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:15.835989    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"]
Nov 03 22:50:15.836260 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:15.836214    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq" podUID="a2122b1d-bbd5-4528-ba52-bcc9fb43897b" containerName="maas-api" containerID="cri-o://51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477" gracePeriod=30
Nov 03 22:50:16.109762 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:50:16.080724    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:50:16.224718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.217745    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:50:16.304697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.298201    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-openshift-service-ca\") pod \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") "
Nov 03 22:50:16.304697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.298500    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-metrics-certs\") pod \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") "
Nov 03 22:50:16.304697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.298624    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-kube-api-access-qdnvr\" (UniqueName: \"kubernetes.io/projected/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-kube-api-access-qdnvr\") pod \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") "
Nov 03 22:50:16.304697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.298777    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls\") pod \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\" (UID: \"a2122b1d-bbd5-4528-ba52-bcc9fb43897b\") "
Nov 03 22:50:16.304697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.298960    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-openshift-service-ca" pod "a2122b1d-bbd5-4528-ba52-bcc9fb43897b" (UID: "a2122b1d-bbd5-4528-ba52-bcc9fb43897b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:50:16.304697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.299839    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:16.364739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.360893    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-metrics-certs" pod "a2122b1d-bbd5-4528-ba52-bcc9fb43897b" (UID: "a2122b1d-bbd5-4528-ba52-bcc9fb43897b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:50:16.410699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.407002    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls" pod "a2122b1d-bbd5-4528-ba52-bcc9fb43897b" (UID: "a2122b1d-bbd5-4528-ba52-bcc9fb43897b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:50:16.430694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.415095    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:16.430694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.415137    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:16.459008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.452916    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-kube-api-access-qdnvr" pod "a2122b1d-bbd5-4528-ba52-bcc9fb43897b" (UID: "a2122b1d-bbd5-4528-ba52-bcc9fb43897b"). InnerVolumeSpecName "kube-api-access-qdnvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:16.459008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.453252    5647 generic.go:356] "Generic (PLEG): container finished" podID="a2122b1d-bbd5-4528-ba52-bcc9fb43897b" containerID="51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477" exitCode=0
Nov 03 22:50:16.459008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.453527    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq" event={"ID":"a2122b1d-bbd5-4528-ba52-bcc9fb43897b","Type":"ContainerDied","Data":"51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477"}
Nov 03 22:50:16.459008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.453590    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq" event={"ID":"a2122b1d-bbd5-4528-ba52-bcc9fb43897b","Type":"ContainerDied","Data":"59da2a194f7171fb4141e69a45d24e80b6d260b67778f6890ff42e1ed43515ea"}
Nov 03 22:50:16.459008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.453726    5647 scope.go:122] "RemoveContainer" containerID="51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477"
Nov 03 22:50:16.459008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.453966    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"
Nov 03 22:50:16.518930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.517224    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdnvr\" (UniqueName: \"kubernetes.io/projected/a2122b1d-bbd5-4528-ba52-bcc9fb43897b-kube-api-access-qdnvr\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:16.554840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.532860    5647 scope.go:122] "RemoveContainer" containerID="51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477"
Nov 03 22:50:16.554840 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:50:16.536845    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477\": container with ID starting with 51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477 not found: ID does not exist" containerID="51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477"
Nov 03 22:50:16.554840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.536889    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477"} err="failed to get container status \"51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477\": rpc error: code = NotFound desc = could not find container \"51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477\": container with ID starting with 51263b5cdba54bc08bcdcf94a8d1fff9dbad15522a6bb82ce8e860844e45d477 not found: ID does not exist"
Nov 03 22:50:16.669265 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.669174    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"]
Nov 03 22:50:16.674368 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.674333    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-26b8d994-b5977797-2q4bq"]
Nov 03 22:50:16.699253 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.699223    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2122b1d-bbd5-4528-ba52-bcc9fb43897b" path="/var/lib/kubelet/pods/a2122b1d-bbd5-4528-ba52-bcc9fb43897b/volumes"
Nov 03 22:50:16.699805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:16.699778    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa9bb1d9-22e7-4de7-876f-e72afb5f485a" path="/var/lib/kubelet/pods/fa9bb1d9-22e7-4de7-876f-e72afb5f485a/volumes"
Nov 03 22:50:17.011695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.007209    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"]
Nov 03 22:50:17.011695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.008531    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a2122b1d-bbd5-4528-ba52-bcc9fb43897b" containerName="maas-api"
Nov 03 22:50:17.011695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.008550    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2122b1d-bbd5-4528-ba52-bcc9fb43897b" containerName="maas-api"
Nov 03 22:50:17.011695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.008877    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2122b1d-bbd5-4528-ba52-bcc9fb43897b" containerName="maas-api"
Nov 03 22:50:17.027705 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.027649    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"]
Nov 03 22:50:17.027938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.027803    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.146241 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.146198    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ebffae22-openshift-default-dockercfg-hq996\""
Nov 03 22:50:17.179384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.179322    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-488h9\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-kube-api-access-488h9\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.179845 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.179813    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-istio-token\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.180167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.180106    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-data\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.180375 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.180358    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4d940d-10b6-41f2-a286-937c804985c4-istiod-ca-cert\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.180585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.180570    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-credential-socket\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.180891 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.180873    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-certs\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.181095 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.181045    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4d940d-10b6-41f2-a286-937c804985c4-istio-podinfo\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.181217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.181202    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-envoy\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.181386 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.181353    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-socket\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283360 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283264    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-488h9\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-kube-api-access-488h9\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283360 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283351    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-istio-token\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283396    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-data\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283439    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4d940d-10b6-41f2-a286-937c804985c4-istiod-ca-cert\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283469    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-credential-socket\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283563    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-certs\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283579    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4d940d-10b6-41f2-a286-937c804985c4-istio-podinfo\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283604    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-envoy\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283636    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-socket\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283875    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-data\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283929    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-credential-socket\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.283991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.283990    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-certs\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.284196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.284124    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-socket\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.284243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.284221    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4d940d-10b6-41f2-a286-937c804985c4-istiod-ca-cert\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.285997 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.285971    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-envoy\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.286300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.286283    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4d940d-10b6-41f2-a286-937c804985c4-istio-podinfo\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.344143 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.344105    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-istio-token\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.344291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.344172    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-488h9\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-kube-api-access-488h9\") pod \"e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") " pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:17.398302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:17.398267    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:18.238291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.238258    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"]
Nov 03 22:50:18.245752 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:50:18.242924    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4d940d_10b6_41f2_a286_937c804985c4.slice/crio-4a2d1a6e41e1151da244550c190892beb7c2534e9307b0cdf706a9997968aaf3 WatchSource:0}: Error finding container 4a2d1a6e41e1151da244550c190892beb7c2534e9307b0cdf706a9997968aaf3: Status 404 returned error can't find the container with id 4a2d1a6e41e1151da244550c190892beb7c2534e9307b0cdf706a9997968aaf3
Nov 03 22:50:18.246181 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.246148    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:18.246252 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.246227    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:18.246313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.246283    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:18.533712 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.533558    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" event={"ID":"6f4d940d-10b6-41f2-a286-937c804985c4","Type":"ContainerStarted","Data":"41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b"}
Nov 03 22:50:18.533928 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.533710    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" event={"ID":"6f4d940d-10b6-41f2-a286-937c804985c4","Type":"ContainerStarted","Data":"4a2d1a6e41e1151da244550c190892beb7c2534e9307b0cdf706a9997968aaf3"}
Nov 03 22:50:18.656744 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.656654    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podStartSLOduration=2.656634861 podStartE2EDuration="2.656634861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:50:18.652297837 +0000 UTC m=+1642.092835097" watchObservedRunningTime="2026-11-03 22:50:18.656634861 +0000 UTC m=+1642.097172121"
Nov 03 22:50:18.728893 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.728852    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"]
Nov 03 22:50:18.729157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.729127    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" containerID="cri-o://2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718" gracePeriod=30
Nov 03 22:50:18.931283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.931137    5647 patch_prober.go:29] interesting pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:50:18.931553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:18.931511    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:50:19.399072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:19.399030    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:19.400933 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:19.400901    5647 patch_prober.go:29] interesting pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 22:50:19.401076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:19.400962    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 22:50:20.430721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:20.403412    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 22:50:20.430721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:20.406856    5647 patch_prober.go:29] interesting pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 22:50:20.430721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:20.406932    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 22:50:20.430721 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:20.421867    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 22:50:21.399411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:21.399377    5647 patch_prober.go:29] interesting pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 22:50:21.399598 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:21.399429    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 22:50:22.409123 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.409088    5647 patch_prober.go:29] interesting pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 22:50:22.409756 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.409715    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 22:50:22.429727 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.427072    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"]
Nov 03 22:50:22.429727 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.429166    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" containerID="cri-o://cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb" gracePeriod=30
Nov 03 22:50:22.433816 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.433775    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"]
Nov 03 22:50:22.434244 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.434211    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy" containerID="cri-o://e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517" gracePeriod=30
Nov 03 22:50:22.508548 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.508051    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"]
Nov 03 22:50:22.508548 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.508301    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz" podUID="7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" containerName="maas-api" containerID="cri-o://73dde6e7dc6dd56648fd3ee5355e1018f371535311caafeb2ebb14fa1ccbeb8b" gracePeriod=30
Nov 03 22:50:22.630274 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.630239    5647 generic.go:356] "Generic (PLEG): container finished" podID="7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" containerID="73dde6e7dc6dd56648fd3ee5355e1018f371535311caafeb2ebb14fa1ccbeb8b" exitCode=0
Nov 03 22:50:22.630394 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:22.630309    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz" event={"ID":"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf","Type":"ContainerDied","Data":"73dde6e7dc6dd56648fd3ee5355e1018f371535311caafeb2ebb14fa1ccbeb8b"}
Nov 03 22:50:23.252700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.235979    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:50:23.473833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.404504    5647 patch_prober.go:29] interesting pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 22:50:23.509910 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.497999    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 22:50:23.599627 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.599582    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-maas-api-tls\") pod \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") "
Nov 03 22:50:23.619313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.619274    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-openshift-service-ca\") pod \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") "
Nov 03 22:50:23.626148 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.625986    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-openshift-service-ca" pod "7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" (UID: "7bee21e7-1482-44e3-8fb1-5816ce1cc2bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:50:23.638009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.636967    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-kube-api-access-qddsc\" (UniqueName: \"kubernetes.io/projected/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-kube-api-access-qddsc\") pod \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") "
Nov 03 22:50:23.638009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.637217    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-metrics-certs\") pod \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\" (UID: \"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf\") "
Nov 03 22:50:23.649767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.638687    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:23.649767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.639815    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-maas-api-tls" pod "7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" (UID: "7bee21e7-1482-44e3-8fb1-5816ce1cc2bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:50:23.661765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.660152    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-metrics-certs" pod "7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" (UID: "7bee21e7-1482-44e3-8fb1-5816ce1cc2bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:50:23.661765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.660341    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-kube-api-access-qddsc" pod "7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" (UID: "7bee21e7-1482-44e3-8fb1-5816ce1cc2bf"). InnerVolumeSpecName "kube-api-access-qddsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:23.843008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.842909    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:23.843008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.842958    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:23.843008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.842973    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qddsc\" (UniqueName: \"kubernetes.io/projected/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf-kube-api-access-qddsc\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:23.896706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.895275    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 03 22:50:23.897773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.897643    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:50:23.928567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.928525    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Nov 03 22:50:23.990693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.957547    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz" event={"ID":"7bee21e7-1482-44e3-8fb1-5816ce1cc2bf","Type":"ContainerDied","Data":"a1e64b01d2edeb3139b864d8c1545c7d1e5bd1e171ed40c830e0770041332e32"}
Nov 03 22:50:23.990693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.957649    5647 scope.go:122] "RemoveContainer" containerID="73dde6e7dc6dd56648fd3ee5355e1018f371535311caafeb2ebb14fa1ccbeb8b"
Nov 03 22:50:23.990693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:23.957925    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"
Nov 03 22:50:24.312108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.312018    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"]
Nov 03 22:50:24.317090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.317056    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz"]
Nov 03 22:50:24.362926 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.362901    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:50:24.366267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366238    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-socket\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.366403 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366280    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-data\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.366403 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366340    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-certs\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.366494 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366436    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-token\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-token\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.366494 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366476    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-podinfo\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.366494 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366477    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-socket" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:24.366650 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366511    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-envoy\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.366985 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366614    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-data" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:24.367060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366724    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-certs" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:24.367060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366684    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istiod-ca-cert\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.367161 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.366956    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istiod-ca-cert" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:50:24.367214 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.367156    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-kube-api-access-4jcsg\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-kube-api-access-4jcsg\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.367214 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.367200    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-credential-socket\") pod \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\" (UID: \"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa\") "
Nov 03 22:50:24.367489 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.367457    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-credential-socket" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:24.367833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.367814    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.367922 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.367838    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.367922 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.367852    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.367922 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.367865    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.367922 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.367879    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.369470 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.369448    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-podinfo" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:50:24.369535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.369516    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-token" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:24.369688 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.369649    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-envoy" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:24.369870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.369851    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-kube-api-access-4jcsg" pod "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" (UID: "e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa"). InnerVolumeSpecName "kube-api-access-4jcsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:24.399391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.399348    5647 patch_prober.go:29] interesting pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 22:50:24.399597 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.399405    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 22:50:24.469800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.469763    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jcsg\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-kube-api-access-4jcsg\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.469800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.469791    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.469800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.469803    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.469800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.469811    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:24.708065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:24.708031    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" path="/var/lib/kubelet/pods/7bee21e7-1482-44e3-8fb1-5816ce1cc2bf/volumes"
Nov 03 22:50:25.015523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.015480    5647 generic.go:356] "Generic (PLEG): container finished" podID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerID="2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718" exitCode=0
Nov 03 22:50:25.015704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.015648    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" event={"ID":"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa","Type":"ContainerDied","Data":"2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718"}
Nov 03 22:50:25.015822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.015718    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8" event={"ID":"e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa","Type":"ContainerDied","Data":"0188be2941a3466361c925687f9442e12386b717d54ac2ed3fbb452629f51d37"}
Nov 03 22:50:25.015822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.015752    5647 scope.go:122] "RemoveContainer" containerID="2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718"
Nov 03 22:50:25.015933 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.015864    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"
Nov 03 22:50:25.039524 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.039502    5647 scope.go:122] "RemoveContainer" containerID="8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822"
Nov 03 22:50:25.054882 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.054860    5647 scope.go:122] "RemoveContainer" containerID="2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718"
Nov 03 22:50:25.055261 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:50:25.055214    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718\": container with ID starting with 2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718 not found: ID does not exist" containerID="2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718"
Nov 03 22:50:25.055313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.055278    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718"} err="failed to get container status \"2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718\": rpc error: code = NotFound desc = could not find container \"2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718\": container with ID starting with 2e8383f3202ad4623086eb18c79c764a25a24afbd2f263a8fea676abb2896718 not found: ID does not exist"
Nov 03 22:50:25.055313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.055304    5647 scope.go:122] "RemoveContainer" containerID="8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822"
Nov 03 22:50:25.055615 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:50:25.055593    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822\": container with ID starting with 8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822 not found: ID does not exist" containerID="8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822"
Nov 03 22:50:25.056752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.056721    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822"} err="failed to get container status \"8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822\": rpc error: code = NotFound desc = could not find container \"8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822\": container with ID starting with 8b675bacc428ad67b53381870a57ae4ad69c9a3304d392e0bb0bbb0570e6a822 not found: ID does not exist"
Nov 03 22:50:25.251700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.247059    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"]
Nov 03 22:50:25.301279 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.292927    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8"]
Nov 03 22:50:25.409695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.400234    5647 patch_prober.go:29] interesting pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Nov 03 22:50:25.409695 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:25.400324    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Nov 03 22:50:26.163637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.163597    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9d7aa682-94c5d86fb-5rmqz" podUID="7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.102:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:50:26.556697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.555071    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:26.556930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.556808    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:26.578694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.573943    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:50:26.715289 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.715244    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" path="/var/lib/kubelet/pods/e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa/volumes"
Nov 03 22:50:26.823220 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.823136    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"]
Nov 03 22:50:26.823895 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.823881    5647 cpu_manager.go:414] "containerMap: removing container" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy"
Nov 03 22:50:26.823944 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.823898    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy"
Nov 03 22:50:26.823944 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.823936    5647 cpu_manager.go:414] "containerMap: removing container" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy"
Nov 03 22:50:26.823944 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.823940    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy"
Nov 03 22:50:26.824036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.823969    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" containerName="maas-api"
Nov 03 22:50:26.824036 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.823974    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" containerName="maas-api"
Nov 03 22:50:26.824103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.824094    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy"
Nov 03 22:50:26.824140 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.824109    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bee21e7-1482-44e3-8fb1-5816ce1cc2bf" containerName="maas-api"
Nov 03 22:50:26.824140 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.824117    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="e39c73d8-a10c-44f5-98d8-a0ccf1a4f1fa" containerName="istio-proxy"
Nov 03 22:50:26.829184 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.829164    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.883889 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.883857    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-de5715-openshift-default-dockercfg-jfm84\""
Nov 03 22:50:26.891940 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.891739    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"]
Nov 03 22:50:26.916220 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916190    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-certs\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.916413 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916250    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-envoy\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.916489 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916401    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63034fce-49e9-40a2-ae8b-818725c20288-istio-podinfo\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.916572 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916550    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlxch\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-kube-api-access-rlxch\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.916650 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916634    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-credential-socket\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.916786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916714    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63034fce-49e9-40a2-ae8b-818725c20288-istiod-ca-cert\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.916858 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916815    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-socket\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.916858 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916849    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-istio-token\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:26.916968 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:26.916908    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-data\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.036811 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.028902    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63034fce-49e9-40a2-ae8b-818725c20288-istiod-ca-cert\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.037281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.037245    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-socket\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.037462 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.037446    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-istio-token\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.037624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.037606    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-data\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.046030 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.046000    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-certs\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.046405 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.046380    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-envoy\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.046746 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.046725    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63034fce-49e9-40a2-ae8b-818725c20288-istio-podinfo\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.047573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.047519    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlxch\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-kube-api-access-rlxch\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.047825 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.047805    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-credential-socket\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.048700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.047055    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63034fce-49e9-40a2-ae8b-818725c20288-istiod-ca-cert\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.076088 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.038899    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-socket\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.076088 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.076064    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-data\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.093701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.092997    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-certs\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.093701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.093261    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-credential-socket\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.131114 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.131074    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-envoy\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.140854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.133778    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-istio-token\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.140854 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.137126    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63034fce-49e9-40a2-ae8b-818725c20288-istio-podinfo\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.155470 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.155435    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlxch\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-kube-api-access-rlxch\") pod \"e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") " pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.447986 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.447897    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:27.485693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.485293    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Nov 03 22:50:27.485693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.485375    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Nov 03 22:50:27.801998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.801973    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:50:27.805575 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.805553    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:50:27.904853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.904822    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-kube-api-access-sw6qk\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-kube-api-access-sw6qk\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.904890    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istiod-ca-cert\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.904940    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-socket\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.905041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.904961    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-kube-api-access-vzrj7\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-kube-api-access-vzrj7\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.905041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905015    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-socket\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905040    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-podinfo\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905069    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-data\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905098    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-podinfo\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.905315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905220    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-envoy\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.905315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905232    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-socket" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.905315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905253    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-token\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-token\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905377    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istiod-ca-cert" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:50:27.905573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905381    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-socket" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.905573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905384    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-envoy\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905573 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905537    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istiod-ca-cert\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.905822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905653    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-certs\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905754    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-credential-socket\") pod \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\" (UID: \"b3881c5b-b563-45f1-9609-8ba6cc8fb68a\") "
Nov 03 22:50:27.905822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.905793    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-token\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-token\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.906095 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.906074    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-credential-socket\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.906364 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.906345    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-data\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.906509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.906494    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-certs\") pod \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\" (UID: \"2b3b44d3-d34b-4837-bd03-ade32eb4a473\") "
Nov 03 22:50:27.907659 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.907637    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:27.907829 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.907814    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:27.907921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.907909    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:27.908115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.908089    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-data" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.908222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.908203    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-credential-socket" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.909162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.908317    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-certs" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.909162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.908490    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-credential-socket" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.909162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.908507    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-certs" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.909162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.908756    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istiod-ca-cert" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:50:27.909162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.908840    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-podinfo" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:50:27.909162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.908933    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-token" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:27.909162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.909127    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-envoy" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.909162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.909134    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-data" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.909575 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.909487    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-kube-api-access-vzrj7" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "kube-api-access-vzrj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:27.911196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.910319    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-kube-api-access-sw6qk" pod "b3881c5b-b563-45f1-9609-8ba6cc8fb68a" (UID: "b3881c5b-b563-45f1-9609-8ba6cc8fb68a"). InnerVolumeSpecName "kube-api-access-sw6qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:27.911196 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.911070    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-podinfo" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:50:27.911704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.911650    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-envoy" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:50:27.912083 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.912048    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-token" pod "2b3b44d3-d34b-4837-bd03-ade32eb4a473" (UID: "2b3b44d3-d34b-4837-bd03-ade32eb4a473"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:50:27.913968 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:50:27.913948    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63034fce_49e9_40a2_ae8b_818725c20288.slice/crio-ef9d0bc3901673596ee5548edffc6344ecfc9bead012fba55e911061cf0922ec WatchSource:0}: Error finding container ef9d0bc3901673596ee5548edffc6344ecfc9bead012fba55e911061cf0922ec: Status 404 returned error can't find the container with id ef9d0bc3901673596ee5548edffc6344ecfc9bead012fba55e911061cf0922ec
Nov 03 22:50:27.914376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.914358    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"]
Nov 03 22:50:27.916682 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.916640    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:27.916758 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.916741    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:27.916797 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:27.916773    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:28.010452 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010412    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010452 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010453    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010470    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010486    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010501    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010514    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010529    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010542    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010556    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw6qk\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-kube-api-access-sw6qk\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010571    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzrj7\" (UniqueName: \"kubernetes.io/projected/2b3b44d3-d34b-4837-bd03-ade32eb4a473-kube-api-access-vzrj7\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010585    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010598    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010611    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010625    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3b44d3-d34b-4837-bd03-ade32eb4a473-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.010743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.010638    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3881c5b-b563-45f1-9609-8ba6cc8fb68a-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:50:28.137951 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.137913    5647 generic.go:356] "Generic (PLEG): container finished" podID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerID="cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb" exitCode=0
Nov 03 22:50:28.138140 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.137975    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"
Nov 03 22:50:28.138200 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.138140    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" event={"ID":"b3881c5b-b563-45f1-9609-8ba6cc8fb68a","Type":"ContainerDied","Data":"cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb"}
Nov 03 22:50:28.138200 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.138191    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr" event={"ID":"b3881c5b-b563-45f1-9609-8ba6cc8fb68a","Type":"ContainerDied","Data":"42840f3ff90c41f62f7189ab64b2d54c60b7667c515933cd1bc070eb3d50548c"}
Nov 03 22:50:28.138306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.138221    5647 scope.go:122] "RemoveContainer" containerID="cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb"
Nov 03 22:50:28.148182 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.148144    5647 generic.go:356] "Generic (PLEG): container finished" podID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerID="e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517" exitCode=0
Nov 03 22:50:28.148283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.148262    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" event={"ID":"2b3b44d3-d34b-4837-bd03-ade32eb4a473","Type":"ContainerDied","Data":"e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517"}
Nov 03 22:50:28.148356 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.148306    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb" event={"ID":"2b3b44d3-d34b-4837-bd03-ade32eb4a473","Type":"ContainerDied","Data":"f3a06798b232a524dbf3af6d04b4ca475b0bb3fea7eefb61baaf283d1274e092"}
Nov 03 22:50:28.148414 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.148386    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"
Nov 03 22:50:28.152974 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.152881    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" event={"ID":"63034fce-49e9-40a2-ae8b-818725c20288","Type":"ContainerStarted","Data":"78d24639b4b8ad9fb47fdadb4300e81f61883b10b802a195d58a4e62e65edf01"}
Nov 03 22:50:28.152974 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.152931    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" event={"ID":"63034fce-49e9-40a2-ae8b-818725c20288","Type":"ContainerStarted","Data":"ef9d0bc3901673596ee5548edffc6344ecfc9bead012fba55e911061cf0922ec"}
Nov 03 22:50:28.160769 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.160744    5647 scope.go:122] "RemoveContainer" containerID="cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb"
Nov 03 22:50:28.161061 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:50:28.161040    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb\": container with ID starting with cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb not found: ID does not exist" containerID="cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb"
Nov 03 22:50:28.161128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.161072    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb"} err="failed to get container status \"cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb\": rpc error: code = NotFound desc = could not find container \"cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb\": container with ID starting with cc389e33aa7490766c36a0d7e405557f2d4d30f3a0f59a2515a93b285f0e28fb not found: ID does not exist"
Nov 03 22:50:28.161128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.161089    5647 scope.go:122] "RemoveContainer" containerID="e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517"
Nov 03 22:50:28.171342 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.171326    5647 scope.go:122] "RemoveContainer" containerID="e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517"
Nov 03 22:50:28.171633 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:50:28.171608    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517\": container with ID starting with e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517 not found: ID does not exist" containerID="e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517"
Nov 03 22:50:28.171796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.171642    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517"} err="failed to get container status \"e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517\": rpc error: code = NotFound desc = could not find container \"e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517\": container with ID starting with e72a40ec352526ef3098f57973589c3e02841bf5a82d2d5b864f9cef3fc7b517 not found: ID does not exist"
Nov 03 22:50:28.312956 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.312857    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podStartSLOduration=2.312844442 podStartE2EDuration="2.312844442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:50:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:50:28.309125862 +0000 UTC m=+1651.749663122" watchObservedRunningTime="2026-11-03 22:50:28.312844442 +0000 UTC m=+1651.753381700"
Nov 03 22:50:28.422794 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.422758    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"]
Nov 03 22:50:28.426431 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.426403    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr"]
Nov 03 22:50:28.471697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.466192    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:28.480706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.479817    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 22:50:28.480706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.479888    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 22:50:28.575819 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.566953    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"]
Nov 03 22:50:28.622955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.620796    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb"]
Nov 03 22:50:28.717770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.717736    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" path="/var/lib/kubelet/pods/2b3b44d3-d34b-4837-bd03-ade32eb4a473/volumes"
Nov 03 22:50:28.718328 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:28.718312    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" path="/var/lib/kubelet/pods/b3881c5b-b563-45f1-9609-8ba6cc8fb68a/volumes"
Nov 03 22:50:29.466008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:29.465979    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:30.290694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:30.290481    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:30.349694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:30.292030    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:50:46.054317 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:50:46.054273    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:50:47.501083 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:47.501000    5647 generic.go:356] "Generic (PLEG): container finished" podID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerID="dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c" exitCode=0
Nov 03 22:50:47.501519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:47.501073    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" event={"ID":"83a20cc8-b64c-42d3-8d53-3a2a04f49735","Type":"ContainerDied","Data":"dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c"}
Nov 03 22:50:47.501570 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:47.501545    5647 scope.go:122] "RemoveContainer" containerID="dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c"
Nov 03 22:50:47.503426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:47.503400    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:47.503504 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:47.503468    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:47.503504 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:47.503499    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:50:48.519923 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:48.519881    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" event={"ID":"83a20cc8-b64c-42d3-8d53-3a2a04f49735","Type":"ContainerStarted","Data":"85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97"}
Nov 03 22:50:49.295778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:49.295491    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:50:49.296000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:49.295781    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 03 22:50:49.296000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:49.295837    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 03 22:50:50.296077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:50.296042    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 03 22:50:50.296505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:50.296100    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 03 22:50:51.296221 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:51.296177    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 03 22:50:51.296835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:51.296260    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 03 22:50:52.296258 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:52.296208    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 03 22:50:52.296767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:52.296271    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 03 22:50:53.299180 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:53.299141    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 03 22:50:53.299733 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:53.299702    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 03 22:50:54.188679 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:54.188637    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:50:54.298273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:54.298247    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:50:54.565872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:50:54.565845    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 22:51:04.771413 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.771370    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"]
Nov 03 22:51:04.772345 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.772322    5647 cpu_manager.go:414] "containerMap: removing container" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy"
Nov 03 22:51:04.772448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.772347    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy"
Nov 03 22:51:04.772448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.772364    5647 cpu_manager.go:414] "containerMap: removing container" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy"
Nov 03 22:51:04.772448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.772371    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy"
Nov 03 22:51:04.772636 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.772622    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b3b44d3-d34b-4837-bd03-ade32eb4a473" containerName="istio-proxy"
Nov 03 22:51:04.772752 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.772643    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3881c5b-b563-45f1-9609-8ba6cc8fb68a" containerName="istio-proxy"
Nov 03 22:51:04.778165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.778142    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.834087 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.834032    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36710485-2249-43c5-aa16-51f95289922d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.834280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.834251    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.834420 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.834401    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.834491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.834442    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szqrn\" (UniqueName: \"kubernetes.io/projected/36710485-2249-43c5-aa16-51f95289922d-kube-api-access-szqrn\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.836151 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.836106    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"]
Nov 03 22:51:04.879396 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.879363    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-4a1480-serving-cert\""
Nov 03 22:51:04.936650 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.936622    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.936830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.936762    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.936830 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.936788    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szqrn\" (UniqueName: \"kubernetes.io/projected/36710485-2249-43c5-aa16-51f95289922d-kube-api-access-szqrn\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.936830 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:04.936794    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-serving-cert: secret "maas-api-e2e-shared-b-w2-4a1480-serving-cert" not found
Nov 03 22:51:04.936958 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:04.936863    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls podName:36710485-2249-43c5-aa16-51f95289922d nodeName:}" failed. No retries permitted until 2026-11-03 22:51:05.436846552 +0000 UTC m=+1688.877383818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls") pod "maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb" (UID: "36710485-2249-43c5-aa16-51f95289922d") : secret "maas-api-e2e-shared-b-w2-4a1480-serving-cert" not found
Nov 03 22:51:04.937045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.937032    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36710485-2249-43c5-aa16-51f95289922d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.937633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.937609    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36710485-2249-43c5-aa16-51f95289922d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.939449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.939428    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:04.998015 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:04.997983    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szqrn\" (UniqueName: \"kubernetes.io/projected/36710485-2249-43c5-aa16-51f95289922d-kube-api-access-szqrn\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:05.454074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:05.454040    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:05.456732 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:05.456711    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:05.700884 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:05.700856    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:06.081765 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:06.081734    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"]
Nov 03 22:51:06.082726 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:51:06.082655    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36710485_2249_43c5_aa16_51f95289922d.slice/crio-dd8993029f8a494a247bf0a3f1552e834a27965e6aa90457fcd82d5e11dad1a6 WatchSource:0}: Error finding container dd8993029f8a494a247bf0a3f1552e834a27965e6aa90457fcd82d5e11dad1a6: Status 404 returned error can't find the container with id dd8993029f8a494a247bf0a3f1552e834a27965e6aa90457fcd82d5e11dad1a6
Nov 03 22:51:06.084438 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:06.084417    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:51:06.683956 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:06.683923    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb" event={"ID":"36710485-2249-43c5-aa16-51f95289922d","Type":"ContainerStarted","Data":"dd8993029f8a494a247bf0a3f1552e834a27965e6aa90457fcd82d5e11dad1a6"}
Nov 03 22:51:07.692615 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:07.692583    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb" event={"ID":"36710485-2249-43c5-aa16-51f95289922d","Type":"ContainerStarted","Data":"2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa"}
Nov 03 22:51:07.693033 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:07.692644    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:07.811887 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:07.811833    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb" podStartSLOduration=2.8978108479999998 podStartE2EDuration="3.811817229s" totalImagesPullingTime="914.006381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:51:07.808493008 +0000 UTC m=+1691.249030267" watchObservedRunningTime="2026-11-03 22:51:07.811817229 +0000 UTC m=+1691.252354488"
Nov 03 22:51:08.264612 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:08.264574    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"]
Nov 03 22:51:08.264852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:08.264828    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy" containerID="cri-o://cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735" gracePeriod=30
Nov 03 22:51:08.315191 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:08.315155    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"]
Nov 03 22:51:08.315417 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:08.315394    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy" containerID="cri-o://77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483" gracePeriod=30
Nov 03 22:51:08.319138 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:08.319112    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"]
Nov 03 22:51:08.319349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:08.319325    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy" containerID="cri-o://2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6" gracePeriod=30
Nov 03 22:51:08.322891 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:08.322862    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"]
Nov 03 22:51:08.323044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:08.323026    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" containerID="cri-o://fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3" gracePeriod=30
Nov 03 22:51:09.046392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:09.045762    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"]
Nov 03 22:51:09.046392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:09.046004    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" containerID="cri-o://64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57" gracePeriod=30
Nov 03 22:51:09.056739 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:09.056421    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"]
Nov 03 22:51:09.056939 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:09.056752    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy" containerID="cri-o://06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3" gracePeriod=30
Nov 03 22:51:09.061153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:09.061130    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"]
Nov 03 22:51:09.061355 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:09.061333    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy" containerID="cri-o://8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e" gracePeriod=30
Nov 03 22:51:11.582206 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:11.582172    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:51:11.582590 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:11.582224    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:51:11.999409 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:11.999373    5647 patch_prober.go:29] interesting pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:51:11.999579 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:11.999434    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:51:13.718982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.718824    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 22:51:13.732867 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.732792    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:51:13.790914 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.790888    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:51:13.795048 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.795020    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:51:13.803911 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.803888    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:51:13.855321 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.855003    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-envoy\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.855321 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.855057    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-socket\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.855321 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.855094    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-data\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.855704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.855650    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-certs\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.855951 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.855918    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-podinfo\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.856110 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856096    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-podinfo\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.856227 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.855695    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-data" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.856312 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856193    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-socket" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.856411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856393    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-certs\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.856581 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856549    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-data\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.856771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856723    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-envoy\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.857538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856799    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-credential-socket\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.857538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856834    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-credential-socket\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.857538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856862    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6104501a-1b6d-4e27-8838-3cc5e45dffee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6104501a-1b6d-4e27-8838-3cc5e45dffee-istiod-ca-cert\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.857538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857110    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-socket\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.857538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857253    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-podinfo\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.857538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857372    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-credential-socket\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.857538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857491    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-token\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-token\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.857538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857532    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-data\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.858090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857590    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-kube-api-access-5gdj8\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-kube-api-access-5gdj8\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.858090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857662    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istiod-ca-cert\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.858090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857735    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-kube-api-access-qvgsz\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-kube-api-access-qvgsz\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.858090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857795    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-kube-api-access-d5jp7\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-kube-api-access-d5jp7\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.858090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857869    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-credential-socket\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.858090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.857961    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-certs\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.858090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858018    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-certs\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.858090 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858054    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istiod-ca-cert\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.858428 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858168    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-envoy\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.858428 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858197    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-data\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.858428 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858270    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-token\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-token\") pod \"6104501a-1b6d-4e27-8838-3cc5e45dffee\" (UID: \"6104501a-1b6d-4e27-8838-3cc5e45dffee\") "
Nov 03 22:51:13.858428 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858344    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-token\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-token\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.858428 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858403    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-envoy\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.858835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858432    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-token\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-token\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.858835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858516    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-socket\") pod \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\" (UID: \"99dbf9ad-841d-4052-ae15-b99f25bb68ca\") "
Nov 03 22:51:13.858835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858591    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istiod-ca-cert\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.858835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858639    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-kube-api-access-w4l8w\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-kube-api-access-w4l8w\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.858835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858743    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-socket\") pod \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\" (UID: \"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c\") "
Nov 03 22:51:13.858835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.858829    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-podinfo\") pod \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\" (UID: \"fbf4759b-d708-48e1-9ac2-7ce04f607b9b\") "
Nov 03 22:51:13.860157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.860133    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.860293 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.860279    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.856410    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-certs" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.860271    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-credential-socket" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.861011    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-certs" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.861194    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-certs" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.861476    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istiod-ca-cert" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.862014    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-certs" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.862096    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-data" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.862445    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istiod-ca-cert" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.862505    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-envoy" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.863234    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-socket" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.863407    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-credential-socket" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.863781    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-podinfo" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.863803    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-data" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.863918    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-credential-socket" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.863983    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6104501a-1b6d-4e27-8838-3cc5e45dffee-istiod-ca-cert" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.865369    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-socket" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.865372    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-socket" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.865612    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-credential-socket" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.865920    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istiod-ca-cert" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:13.866503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.866209    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-data" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.870429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.866609    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:51:13.870429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.866702    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:51:13.870429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.867022    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-token" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:13.870978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.870957    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-kube-api-access-d5jp7" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "kube-api-access-d5jp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:13.872228 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.872087    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-podinfo" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:13.872228 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.872175    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-token" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:13.874099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.873963    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-envoy" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.874099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.874021    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-envoy" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.874099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.874075    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-kube-api-access-w4l8w" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "kube-api-access-w4l8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:13.874312 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.874104    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-token" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:13.874657 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.874419    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-envoy" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:13.874657 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.874475    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-podinfo" pod "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" (UID: "cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:13.874657 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.874531    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-kube-api-access-5gdj8" pod "6104501a-1b6d-4e27-8838-3cc5e45dffee" (UID: "6104501a-1b6d-4e27-8838-3cc5e45dffee"). InnerVolumeSpecName "kube-api-access-5gdj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:13.874657 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.874618    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-token" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:13.875232 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.874257    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-kube-api-access-qvgsz" pod "99dbf9ad-841d-4052-ae15-b99f25bb68ca" (UID: "99dbf9ad-841d-4052-ae15-b99f25bb68ca"). InnerVolumeSpecName "kube-api-access-qvgsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:13.880112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.880081    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-podinfo" pod "fbf4759b-d708-48e1-9ac2-7ce04f607b9b" (UID: "fbf4759b-d708-48e1-9ac2-7ce04f607b9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:13.905725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.905649    5647 generic.go:356] "Generic (PLEG): container finished" podID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerID="cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735" exitCode=0
Nov 03 22:51:13.905889 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.905730    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" event={"ID":"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c","Type":"ContainerDied","Data":"cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735"}
Nov 03 22:51:13.905889 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.905795    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt" event={"ID":"cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c","Type":"ContainerDied","Data":"b092b42beb7f10e0646cc11b18215211d151d3fabcadda093e7bc123fd807826"}
Nov 03 22:51:13.905889 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.905740    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"
Nov 03 22:51:13.906163 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.905823    5647 scope.go:122] "RemoveContainer" containerID="cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735"
Nov 03 22:51:13.907456 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.907435    5647 generic.go:356] "Generic (PLEG): container finished" podID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerID="fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3" exitCode=0
Nov 03 22:51:13.907559 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.907488    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"
Nov 03 22:51:13.907559 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.907528    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" event={"ID":"99dbf9ad-841d-4052-ae15-b99f25bb68ca","Type":"ContainerDied","Data":"fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3"}
Nov 03 22:51:13.907941 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.907581    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl" event={"ID":"99dbf9ad-841d-4052-ae15-b99f25bb68ca","Type":"ContainerDied","Data":"7ac1203b927b5d1c7fa99439a96da88e528a3a4f61c74c730d58f010a3db5e5a"}
Nov 03 22:51:13.909302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.909277    5647 generic.go:356] "Generic (PLEG): container finished" podID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerID="2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6" exitCode=0
Nov 03 22:51:13.909418 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.909381    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" event={"ID":"6104501a-1b6d-4e27-8838-3cc5e45dffee","Type":"ContainerDied","Data":"2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6"}
Nov 03 22:51:13.909418 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.909410    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2" event={"ID":"6104501a-1b6d-4e27-8838-3cc5e45dffee","Type":"ContainerDied","Data":"b56f6240f129258c457c9e6d014b1ce92f9739ccd710c2125b57b1be9fae6155"}
Nov 03 22:51:13.909558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.909469    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"
Nov 03 22:51:13.912304 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.912285    5647 generic.go:356] "Generic (PLEG): container finished" podID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerID="77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483" exitCode=0
Nov 03 22:51:13.912392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.912326    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"
Nov 03 22:51:13.912392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.912336    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" event={"ID":"fbf4759b-d708-48e1-9ac2-7ce04f607b9b","Type":"ContainerDied","Data":"77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483"}
Nov 03 22:51:13.912392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.912372    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx" event={"ID":"fbf4759b-d708-48e1-9ac2-7ce04f607b9b","Type":"ContainerDied","Data":"7b66c1591ca38878adb0dd86597c959a52382d7910d63de8d7c009d2fb6aabbb"}
Nov 03 22:51:13.920305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.920266    5647 scope.go:122] "RemoveContainer" containerID="cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735"
Nov 03 22:51:13.920637 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:13.920615    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735\": container with ID starting with cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735 not found: ID does not exist" containerID="cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735"
Nov 03 22:51:13.920757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.920643    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735"} err="failed to get container status \"cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735\": rpc error: code = NotFound desc = could not find container \"cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735\": container with ID starting with cb45ae545aba4a61176fbc6f60d573aa0a84eb2d7ce208e5e39075945d910735 not found: ID does not exist"
Nov 03 22:51:13.920757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.920658    5647 scope.go:122] "RemoveContainer" containerID="fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3"
Nov 03 22:51:13.932643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.932621    5647 scope.go:122] "RemoveContainer" containerID="fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3"
Nov 03 22:51:13.932957 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:13.932939    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3\": container with ID starting with fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3 not found: ID does not exist" containerID="fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3"
Nov 03 22:51:13.933009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.932964    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3"} err="failed to get container status \"fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3\": rpc error: code = NotFound desc = could not find container \"fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3\": container with ID starting with fad76d22ba65afa4e31c629b713bd6feed255d9ffd201aff1c893fc0d4b8e1c3 not found: ID does not exist"
Nov 03 22:51:13.933009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.932983    5647 scope.go:122] "RemoveContainer" containerID="2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6"
Nov 03 22:51:13.943293 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.943274    5647 scope.go:122] "RemoveContainer" containerID="2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6"
Nov 03 22:51:13.943553 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:13.943535    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6\": container with ID starting with 2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6 not found: ID does not exist" containerID="2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6"
Nov 03 22:51:13.943604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.943566    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6"} err="failed to get container status \"2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6\": rpc error: code = NotFound desc = could not find container \"2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6\": container with ID starting with 2081ab31cb348b874265550e149bc2138cf257a4cee0b1e291a46d5e6fa486b6 not found: ID does not exist"
Nov 03 22:51:13.943604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.943582    5647 scope.go:122] "RemoveContainer" containerID="77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483"
Nov 03 22:51:13.953069 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.953049    5647 scope.go:122] "RemoveContainer" containerID="77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483"
Nov 03 22:51:13.953383 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:13.953354    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483\": container with ID starting with 77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483 not found: ID does not exist" containerID="77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483"
Nov 03 22:51:13.953444 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.953388    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483"} err="failed to get container status \"77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483\": rpc error: code = NotFound desc = could not find container \"77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483\": container with ID starting with 77bf90771e753dc95c61acff0e1a8261875c2ac43983eef43199bcc614001483 not found: ID does not exist"
Nov 03 22:51:13.961937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.961906    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.961941    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.961955    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.961968    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gdj8\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-kube-api-access-5gdj8\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.961981    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.961993    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvgsz\" (UniqueName: \"kubernetes.io/projected/99dbf9ad-841d-4052-ae15-b99f25bb68ca-kube-api-access-qvgsz\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962007    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5jp7\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-kube-api-access-d5jp7\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962021    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962034    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962048    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962061    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962073    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962086    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962098    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962111    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962124    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962137    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962152    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962163    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962176    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4l8w\" (UniqueName: \"kubernetes.io/projected/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-kube-api-access-w4l8w\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962187    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962199    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962213    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962227    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962240    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6104501a-1b6d-4e27-8838-3cc5e45dffee-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962253    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962265    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962278    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962290    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf4759b-d708-48e1-9ac2-7ce04f607b9b-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962302    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99dbf9ad-841d-4052-ae15-b99f25bb68ca-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962315    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962328    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6104501a-1b6d-4e27-8838-3cc5e45dffee-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962343    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6104501a-1b6d-4e27-8838-3cc5e45dffee-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:13.962395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:13.962356    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99dbf9ad-841d-4052-ae15-b99f25bb68ca-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.396297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.396271    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:51:14.457481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.457450    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"]
Nov 03 22:51:14.461161 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.461141    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx"]
Nov 03 22:51:14.470130 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470105    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-kube-api-access-8w5mc\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-kube-api-access-8w5mc\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.470259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470139    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-token\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-token\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.470259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470204    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-podinfo\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.471172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470570    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-socket\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.471172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470684    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-certs\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.471172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470789    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-envoy\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.471172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470881    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-socket" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.471172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470913    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-credential-socket\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.471172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470978    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istiod-ca-cert\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.471172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470995    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-data\") pod \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\" (UID: \"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec\") "
Nov 03 22:51:14.471172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.470994    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-certs" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.471567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.471314    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-credential-socket" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.471567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.471532    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istiod-ca-cert" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:14.471713 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.471693    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-data" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.471978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.471895    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.471978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.471918    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.471978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.471933    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.471978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.471947    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.471978 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.471958    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.473652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.473625    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-podinfo" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:14.473792 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.473753    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-envoy" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.474594 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.474575    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-kube-api-access-8w5mc" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "kube-api-access-8w5mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:14.486052 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.486022    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-token" pod "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" (UID: "89b8beb3-d6b3-449c-8ec2-d7a5fec73aec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:14.495145 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.495125    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:51:14.501584 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.501551    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.580464    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-certs\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.580522    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-kube-api-access-6sq54\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-kube-api-access-6sq54\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.580601    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-token\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-token\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.580709    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-envoy\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.580787    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-kube-api-access-jshnz\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-kube-api-access-jshnz\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.580834    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05719b59-3d32-4f91-b8f1-fa91e1901161-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05719b59-3d32-4f91-b8f1-fa91e1901161-istiod-ca-cert\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.580936    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-socket\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581019    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-podinfo\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581070    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-data\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581110    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-token\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-token\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581134    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-envoy\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581264    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-credential-socket\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581323    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/446dfad7-dee2-44a7-83f7-9cee265d5d24-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/446dfad7-dee2-44a7-83f7-9cee265d5d24-istiod-ca-cert\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581407    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-socket\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581457    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-credential-socket\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581560    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-data\") pod \"05719b59-3d32-4f91-b8f1-fa91e1901161\" (UID: \"05719b59-3d32-4f91-b8f1-fa91e1901161\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581592    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-certs\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.589692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.581629    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-podinfo\") pod \"446dfad7-dee2-44a7-83f7-9cee265d5d24\" (UID: \"446dfad7-dee2-44a7-83f7-9cee265d5d24\") "
Nov 03 22:51:14.603866 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.603228    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-data" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.603866 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.603455    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-certs" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651230    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-kube-api-access-6sq54" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "kube-api-access-6sq54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651289    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"]
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651329    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2"]
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651423    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651441    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sq54\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-kube-api-access-6sq54\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651501    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651515    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651532    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w5mc\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-kube-api-access-8w5mc\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651547    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.651943 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.651561    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.653775    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05719b59-3d32-4f91-b8f1-fa91e1901161-istiod-ca-cert" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.656035    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-kube-api-access-jshnz" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "kube-api-access-jshnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.656179    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-credential-socket" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.656291    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-socket" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.656408    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-credential-socket" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.656515    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-socket" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.656626    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-data" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.656801    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-certs" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.659775    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-podinfo" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.671431    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/446dfad7-dee2-44a7-83f7-9cee265d5d24-istiod-ca-cert" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:51:14.683693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.682123    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-envoy" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.697717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.696999    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-token" pod "05719b59-3d32-4f91-b8f1-fa91e1901161" (UID: "05719b59-3d32-4f91-b8f1-fa91e1901161"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:14.707778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.702812    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-envoy" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:51:14.726692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.723277    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-podinfo" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:51:14.726692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.724703    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-token" pod "446dfad7-dee2-44a7-83f7-9cee265d5d24" (UID: "446dfad7-dee2-44a7-83f7-9cee265d5d24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:51:14.758342 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.758186    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.758856 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.758840    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/446dfad7-dee2-44a7-83f7-9cee265d5d24-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.758991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.758981    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759060 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759053    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759123 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.758778    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" path="/var/lib/kubelet/pods/6104501a-1b6d-4e27-8838-3cc5e45dffee/volumes"
Nov 03 22:51:14.759805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759112    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759815    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" path="/var/lib/kubelet/pods/fbf4759b-d708-48e1-9ac2-7ce04f607b9b/volumes"
Nov 03 22:51:14.759860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759822    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759860 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759841    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759857    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759873    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759887    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jshnz\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-kube-api-access-jshnz\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759902    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05719b59-3d32-4f91-b8f1-fa91e1901161-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759919    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/446dfad7-dee2-44a7-83f7-9cee265d5d24-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759937    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759951    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/446dfad7-dee2-44a7-83f7-9cee265d5d24-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.759971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.759966    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05719b59-3d32-4f91-b8f1-fa91e1901161-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:51:14.761068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.760895    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"]
Nov 03 22:51:14.761068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.760938    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl"]
Nov 03 22:51:14.817856 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.817823    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"]
Nov 03 22:51:14.821612 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.821588    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt"]
Nov 03 22:51:14.921586 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.921493    5647 generic.go:356] "Generic (PLEG): container finished" podID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerID="06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3" exitCode=0
Nov 03 22:51:14.921586 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.921544    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"
Nov 03 22:51:14.921855 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.921585    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" event={"ID":"446dfad7-dee2-44a7-83f7-9cee265d5d24","Type":"ContainerDied","Data":"06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3"}
Nov 03 22:51:14.921855 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.921642    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp" event={"ID":"446dfad7-dee2-44a7-83f7-9cee265d5d24","Type":"ContainerDied","Data":"c63aeb6feb77c3f2aa92f4710c775f9b25f49ecfd36b6fc60f694a7ecd3bb1d7"}
Nov 03 22:51:14.921855 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.921710    5647 scope.go:122] "RemoveContainer" containerID="06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3"
Nov 03 22:51:14.926750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.926728    5647 generic.go:356] "Generic (PLEG): container finished" podID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerID="8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e" exitCode=0
Nov 03 22:51:14.926886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.926776    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"
Nov 03 22:51:14.926886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.926808    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" event={"ID":"05719b59-3d32-4f91-b8f1-fa91e1901161","Type":"ContainerDied","Data":"8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e"}
Nov 03 22:51:14.926886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.926844    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr" event={"ID":"05719b59-3d32-4f91-b8f1-fa91e1901161","Type":"ContainerDied","Data":"f70a9b60a6e1a901cea28421aed7a6c4810a793aab210c515fc16ccf6dbd15b6"}
Nov 03 22:51:14.928344 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.928318    5647 generic.go:356] "Generic (PLEG): container finished" podID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerID="64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57" exitCode=0
Nov 03 22:51:14.928448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.928347    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" event={"ID":"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec","Type":"ContainerDied","Data":"64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57"}
Nov 03 22:51:14.928448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.928366    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"
Nov 03 22:51:14.928448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.928387    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz" event={"ID":"89b8beb3-d6b3-449c-8ec2-d7a5fec73aec","Type":"ContainerDied","Data":"f88d11de488f437b59e8c6b8dae7bde51c6f326910b6c0e9cb8cb467278b40a7"}
Nov 03 22:51:14.933462 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.933446    5647 scope.go:122] "RemoveContainer" containerID="06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3"
Nov 03 22:51:14.933776 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:14.933753    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3\": container with ID starting with 06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3 not found: ID does not exist" containerID="06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3"
Nov 03 22:51:14.933870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.933785    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3"} err="failed to get container status \"06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3\": rpc error: code = NotFound desc = could not find container \"06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3\": container with ID starting with 06d6453f63438a5a13a34582b89041dc477f95e25bf068f6117e16c5bcb809e3 not found: ID does not exist"
Nov 03 22:51:14.933870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.933805    5647 scope.go:122] "RemoveContainer" containerID="8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e"
Nov 03 22:51:14.950006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.949979    5647 scope.go:122] "RemoveContainer" containerID="8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e"
Nov 03 22:51:14.950652 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:14.950626    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e\": container with ID starting with 8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e not found: ID does not exist" containerID="8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e"
Nov 03 22:51:14.950853 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.950825    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e"} err="failed to get container status \"8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e\": rpc error: code = NotFound desc = could not find container \"8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e\": container with ID starting with 8fe33ae090ca8fbf41ab62ece9000262e1e832118b1eb854154490e53bd91a1e not found: ID does not exist"
Nov 03 22:51:14.950962 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:14.950949    5647 scope.go:122] "RemoveContainer" containerID="64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57"
Nov 03 22:51:15.104379 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.104349    5647 scope.go:122] "RemoveContainer" containerID="64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57"
Nov 03 22:51:15.104830 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:15.104805    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57\": container with ID starting with 64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57 not found: ID does not exist" containerID="64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57"
Nov 03 22:51:15.105013 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.104994    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57"} err="failed to get container status \"64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57\": rpc error: code = NotFound desc = could not find container \"64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57\": container with ID starting with 64b9283e055b3a29cd88da0d4cbb49cacdc676ed3b06846ca33baef70d034f57 not found: ID does not exist"
Nov 03 22:51:15.339775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.339724    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"]
Nov 03 22:51:15.345377 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.345341    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp"]
Nov 03 22:51:15.445528 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.445488    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"]
Nov 03 22:51:15.448588 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.448561    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr"]
Nov 03 22:51:15.557770 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.557691    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"]
Nov 03 22:51:15.560994 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.560971    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz"]
Nov 03 22:51:15.989558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:15.989529    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:51:16.043717 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:16.043641    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:51:16.698291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:16.698258    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" path="/var/lib/kubelet/pods/05719b59-3d32-4f91-b8f1-fa91e1901161/volumes"
Nov 03 22:51:16.698833 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:16.698818    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" path="/var/lib/kubelet/pods/446dfad7-dee2-44a7-83f7-9cee265d5d24/volumes"
Nov 03 22:51:16.699270 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:16.699258    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" path="/var/lib/kubelet/pods/89b8beb3-d6b3-449c-8ec2-d7a5fec73aec/volumes"
Nov 03 22:51:16.700074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:16.700058    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" path="/var/lib/kubelet/pods/99dbf9ad-841d-4052-ae15-b99f25bb68ca/volumes"
Nov 03 22:51:16.700500 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:16.700487    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" path="/var/lib/kubelet/pods/cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c/volumes"
Nov 03 22:51:23.816179 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816144    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"]
Nov 03 22:51:23.816886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816867    5647 cpu_manager.go:414] "containerMap: removing container" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy"
Nov 03 22:51:23.816886 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816885    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816900    5647 cpu_manager.go:414] "containerMap: removing container" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816905    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816919    5647 cpu_manager.go:414] "containerMap: removing container" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816924    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816946    5647 cpu_manager.go:414] "containerMap: removing container" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816951    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816962    5647 cpu_manager.go:414] "containerMap: removing container" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816967    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816975    5647 cpu_manager.go:414] "containerMap: removing container" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816979    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816990    5647 cpu_manager.go:414] "containerMap: removing container" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy"
Nov 03 22:51:23.816993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.816995    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy"
Nov 03 22:51:23.817325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.817106    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="446dfad7-dee2-44a7-83f7-9cee265d5d24" containerName="istio-proxy"
Nov 03 22:51:23.817325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.817115    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="05719b59-3d32-4f91-b8f1-fa91e1901161" containerName="istio-proxy"
Nov 03 22:51:23.817325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.817121    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="99dbf9ad-841d-4052-ae15-b99f25bb68ca" containerName="istio-proxy"
Nov 03 22:51:23.817325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.817131    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbf4759b-d708-48e1-9ac2-7ce04f607b9b" containerName="istio-proxy"
Nov 03 22:51:23.817325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.817138    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf80ffbd-f32e-4b7c-93a6-64ea5a661b2c" containerName="istio-proxy"
Nov 03 22:51:23.817325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.817145    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="6104501a-1b6d-4e27-8838-3cc5e45dffee" containerName="istio-proxy"
Nov 03 22:51:23.817325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.817151    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="89b8beb3-d6b3-449c-8ec2-d7a5fec73aec" containerName="istio-proxy"
Nov 03 22:51:23.822439 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.822420    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.874562 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.874519    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-433db8-openshift-default-dockercfg-v7m87\""
Nov 03 22:51:23.927955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.927926    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"]
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.976850    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-certs\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.976999    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-envoy\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.977033    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgmbk\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-kube-api-access-bgmbk\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.977060    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-data\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.977100    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-token\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.977170    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-credential-socket\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.977232    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-socket\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.977305    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-podinfo\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:23.978720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:23.977334    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7424a3-4b15-44af-beae-c41b3eb13801-istiod-ca-cert\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.091266 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.088832    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-envoy\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.091440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.091253    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgmbk\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-kube-api-access-bgmbk\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.091440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.091295    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-data\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.091440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.091336    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-token\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.091440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.091407    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-credential-socket\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.091655 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.091605    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-socket\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.091764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.091747    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-podinfo\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.091834 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.091780    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7424a3-4b15-44af-beae-c41b3eb13801-istiod-ca-cert\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.092055 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.092028    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-data\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.092558 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.092523    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-credential-socket\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.092840 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.092819    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7424a3-4b15-44af-beae-c41b3eb13801-istiod-ca-cert\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.092945 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.092923    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-socket\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.093250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.093044    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-certs\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.093380 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.093210    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-certs\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.097263 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.097237    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-podinfo\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.097723 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.097654    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-envoy\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.176699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.175492    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-token\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.203309 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.202905    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgmbk\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-kube-api-access-bgmbk\") pod \"e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") " pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.436162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.436073    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:24.869715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.869573    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"]
Nov 03 22:51:24.872652 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:51:24.872619    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7424a3_4b15_44af_beae_c41b3eb13801.slice/crio-567f83b6fe6c96bae0fa34f799a222af045646df8f54315127d0a541a22ac093 WatchSource:0}: Error finding container 567f83b6fe6c96bae0fa34f799a222af045646df8f54315127d0a541a22ac093: Status 404 returned error can't find the container with id 567f83b6fe6c96bae0fa34f799a222af045646df8f54315127d0a541a22ac093
Nov 03 22:51:24.875623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.875569    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:51:24.875743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.875660    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:51:24.875743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:24.875739    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:51:25.112246 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:25.112210    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" event={"ID":"ea7424a3-4b15-44af-beae-c41b3eb13801","Type":"ContainerStarted","Data":"ed3507663c10e41044028ea6347525817e38e35ae49788fbca0fe06741a978b8"}
Nov 03 22:51:25.112426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:25.112274    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" event={"ID":"ea7424a3-4b15-44af-beae-c41b3eb13801","Type":"ContainerStarted","Data":"567f83b6fe6c96bae0fa34f799a222af045646df8f54315127d0a541a22ac093"}
Nov 03 22:51:25.237908 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:25.237856    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podStartSLOduration=2.237840801 podStartE2EDuration="2.237840801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:51:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:51:25.236570371 +0000 UTC m=+1708.677107642" watchObservedRunningTime="2026-11-03 22:51:25.237840801 +0000 UTC m=+1708.678378060"
Nov 03 22:51:25.437692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:25.436973    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:25.457120 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:25.455543    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:26.119277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:26.119240    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:26.120223 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:26.120206    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:51:30.662529 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.662499    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"]
Nov 03 22:51:30.667157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.667136    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.721392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.721365    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-ea1be0-serving-cert\""
Nov 03 22:51:30.774717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.774683    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"]
Nov 03 22:51:30.814399 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.814367    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.814566 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.814415    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.814566 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.814500    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.814640 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.814621    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42brw\" (UniqueName: \"kubernetes.io/projected/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-kube-api-access-42brw\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.916538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.916446    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.916538 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.916493    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.916817 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:30.916585    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-serving-cert: secret "maas-api-e2e-shared-a-w0-ea1be0-serving-cert" not found
Nov 03 22:51:30.916817 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.916609    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.916817 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:30.916655    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls podName:f2a20bfd-e59f-44ad-9463-6d3d54ef713b nodeName:}" failed. No retries permitted until 2026-11-03 22:51:31.416638843 +0000 UTC m=+1714.857176083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls") pod "maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc" (UID: "f2a20bfd-e59f-44ad-9463-6d3d54ef713b") : secret "maas-api-e2e-shared-a-w0-ea1be0-serving-cert" not found
Nov 03 22:51:30.916817 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.916767    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42brw\" (UniqueName: \"kubernetes.io/projected/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-kube-api-access-42brw\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.917190 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.917167    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:30.919141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:30.919117    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:31.005961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.002640    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42brw\" (UniqueName: \"kubernetes.io/projected/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-kube-api-access-42brw\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:31.538708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.533723    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:31.565071 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.565025    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:31.583468 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.582567    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:31.693029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.692987    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"]
Nov 03 22:51:31.707037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.706998    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:31.805813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.797457    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"]
Nov 03 22:51:31.805813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.798744    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-433db8\""
Nov 03 22:51:31.825710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.820940    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-433db8-dockercfg-xr684\""
Nov 03 22:51:31.942700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.908261    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk\" (UID: \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:31.985021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:31.984974    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gnd7\" (UniqueName: \"kubernetes.io/projected/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-kube-api-access-9gnd7\") pod \"payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk\" (UID: \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:32.150199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:32.134568    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk\" (UID: \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:32.150199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:32.134641    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnd7\" (UniqueName: \"kubernetes.io/projected/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-kube-api-access-9gnd7\") pod \"payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk\" (UID: \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:32.150199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:32.136205    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk\" (UID: \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:32.218819 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:32.218779    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gnd7\" (UniqueName: \"kubernetes.io/projected/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-kube-api-access-9gnd7\") pod \"payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk\" (UID: \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:32.372101 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:32.372070    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:32.525584 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:32.525549    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"]
Nov 03 22:51:32.525939 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:51:32.525913    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a20bfd_e59f_44ad_9463_6d3d54ef713b.slice/crio-99403e6dee1387fe8d28283c157705012bf4d3145dee94fa165a58af64f99e52 WatchSource:0}: Error finding container 99403e6dee1387fe8d28283c157705012bf4d3145dee94fa165a58af64f99e52: Status 404 returned error can't find the container with id 99403e6dee1387fe8d28283c157705012bf4d3145dee94fa165a58af64f99e52
Nov 03 22:51:32.761012 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:32.760988    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"]
Nov 03 22:51:32.761598 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:51:32.761569    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89fbd4d3_1c94_49e4_b81c_2f0fb8da3705.slice/crio-58d15305b1a44a58101faf747ce3b4c5d673a9b37d43774834d94190557e5126 WatchSource:0}: Error finding container 58d15305b1a44a58101faf747ce3b4c5d673a9b37d43774834d94190557e5126: Status 404 returned error can't find the container with id 58d15305b1a44a58101faf747ce3b4c5d673a9b37d43774834d94190557e5126
Nov 03 22:51:33.210718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:33.210686    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk" event={"ID":"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705","Type":"ContainerStarted","Data":"4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd"}
Nov 03 22:51:33.210912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:33.210746    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk" event={"ID":"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705","Type":"ContainerStarted","Data":"58d15305b1a44a58101faf747ce3b4c5d673a9b37d43774834d94190557e5126"}
Nov 03 22:51:33.210912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:33.210900    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:33.212099 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:33.212077    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc" event={"ID":"f2a20bfd-e59f-44ad-9463-6d3d54ef713b","Type":"ContainerStarted","Data":"99403e6dee1387fe8d28283c157705012bf4d3145dee94fa165a58af64f99e52"}
Nov 03 22:51:33.331735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:33.331385    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk" podStartSLOduration=2.331364045 podStartE2EDuration="2.331364045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:51:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:51:33.33009885 +0000 UTC m=+1716.770636114" watchObservedRunningTime="2026-11-03 22:51:33.331364045 +0000 UTC m=+1716.771901306"
Nov 03 22:51:34.222104 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:34.222070    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc" event={"ID":"f2a20bfd-e59f-44ad-9463-6d3d54ef713b","Type":"ContainerStarted","Data":"55f8abcb7853da15299e6ad2f46d9fb36e1a4bc93872ced269b3865f53abcc4e"}
Nov 03 22:51:34.338697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:34.338607    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc" podStartSLOduration=3.477377685 podStartE2EDuration="4.338595s" totalImagesPullingTime="861.217315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:51:34.338006311 +0000 UTC m=+1717.778543572" watchObservedRunningTime="2026-11-03 22:51:34.338595 +0000 UTC m=+1717.779132259"
Nov 03 22:51:35.229482 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:35.229453    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:41.242633 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:41.242596    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:51:44.224901 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:44.224870    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 22:51:46.045977 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:51:46.045937    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:51:59.101242 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:51:59.101204    5647 scope.go:122] "RemoveContainer" containerID="3ebf3f8516c733ce735248f34d1f4e5c03917449794a2edfd582e43cf12f7063"
Nov 03 22:52:16.046709 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:52:16.046636    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:52:16.504704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.501956    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"]
Nov 03 22:52:16.557229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.557191    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.600710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.592244    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"]
Nov 03 22:52:16.627113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.623855    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-efce83-openshift-default-dockercfg-chzm5\""
Nov 03 22:52:16.685455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.685405    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928babb1-e87e-4ce6-b257-d149c947c724-istiod-ca-cert\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.685839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.685809    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7bwl\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-kube-api-access-x7bwl\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.686182 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.686155    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-istio-token\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.686340 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.686325    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-envoy\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.686486 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.686471    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-certs\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.686621 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.686606    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-data\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.686779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.686764    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928babb1-e87e-4ce6-b257-d149c947c724-istio-podinfo\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.687047 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.687024    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-socket\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.687311 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.687296    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-credential-socket\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.797075 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.796982    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-credential-socket\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.797346 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.797324    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928babb1-e87e-4ce6-b257-d149c947c724-istiod-ca-cert\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.797484 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.797470    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7bwl\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-kube-api-access-x7bwl\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.797766 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.797746    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-istio-token\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.797904 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.797889    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-envoy\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.798068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.798052    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-certs\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.798159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.798147    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-data\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.798277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.798262    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928babb1-e87e-4ce6-b257-d149c947c724-istio-podinfo\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.798487 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.798465    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-socket\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.801709 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.801383    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-certs\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.801709 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.801476    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-credential-socket\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.852540 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.852245    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-envoy\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.853315 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.853279    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-data\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.895712 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.888883    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928babb1-e87e-4ce6-b257-d149c947c724-istio-podinfo\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.953805 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.918789    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-socket\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:16.967789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:16.967752    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-istio-token\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:17.027242 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.027200    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928babb1-e87e-4ce6-b257-d149c947c724-istiod-ca-cert\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:17.044964 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.044925    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7bwl\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-kube-api-access-x7bwl\") pod \"e2e-api-a-efce83-openshift-default-5b595bb589-8j7df\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") " pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:17.249217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.249179    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:17.686757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.686658    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"]
Nov 03 22:52:17.722859 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:52:17.722822    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928babb1_e87e_4ce6_b257_d149c947c724.slice/crio-170dd9c7069f52225a266dc4316e274d19fc577ed81e2ddaad97dc8842d00dac WatchSource:0}: Error finding container 170dd9c7069f52225a266dc4316e274d19fc577ed81e2ddaad97dc8842d00dac: Status 404 returned error can't find the container with id 170dd9c7069f52225a266dc4316e274d19fc577ed81e2ddaad97dc8842d00dac
Nov 03 22:52:17.725634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.725599    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:52:17.725750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.725706    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:52:17.725750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.725738    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:52:17.917503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.917462    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" event={"ID":"928babb1-e87e-4ce6-b257-d149c947c724","Type":"ContainerStarted","Data":"e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39"}
Nov 03 22:52:17.917696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:17.917533    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" event={"ID":"928babb1-e87e-4ce6-b257-d149c947c724","Type":"ContainerStarted","Data":"170dd9c7069f52225a266dc4316e274d19fc577ed81e2ddaad97dc8842d00dac"}
Nov 03 22:52:18.037716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:18.037613    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podStartSLOduration=2.037595613 podStartE2EDuration="2.037595613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:52:18.037212101 +0000 UTC m=+1761.477749361" watchObservedRunningTime="2026-11-03 22:52:18.037595613 +0000 UTC m=+1761.478132874"
Nov 03 22:52:18.251649 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:18.251611    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:18.253836 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:18.253807    5647 patch_prober.go:29] interesting pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Nov 03 22:52:18.254083 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:18.254046    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Nov 03 22:52:19.254142 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:19.254112    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:19.931471 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:19.931440    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:19.932577 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:19.932555    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:52:28.306340 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:28.306307    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"]
Nov 03 22:52:33.175509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.175471    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:52:33.307510 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.307450    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4" podUID="ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd" containerName="payload-processing" containerID="cri-o://f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346" gracePeriod=25
Nov 03 22:52:33.773698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.773119    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:52:33.794998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.794796    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-plugins-config-volume\") pod \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\" (UID: \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\") "
Nov 03 22:52:33.795875 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.795850    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-kube-api-access-4mc8r\" (UniqueName: \"kubernetes.io/projected/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-kube-api-access-4mc8r\") pod \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\" (UID: \"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd\") "
Nov 03 22:52:33.797182 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.795649    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-plugins-config-volume" pod "ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd" (UID: "ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:52:33.799930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.799894    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-kube-api-access-4mc8r" pod "ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd" (UID: "ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd"). InnerVolumeSpecName "kube-api-access-4mc8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:52:33.906636 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.906525    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:52:33.906636 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:33.906552    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mc8r\" (UniqueName: \"kubernetes.io/projected/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd-kube-api-access-4mc8r\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:52:34.274155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.274121    5647 generic.go:356] "Generic (PLEG): container finished" podID="ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd" containerID="f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346" exitCode=0
Nov 03 22:52:34.274571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.274182    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"
Nov 03 22:52:34.274571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.274206    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4" event={"ID":"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd","Type":"ContainerDied","Data":"f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346"}
Nov 03 22:52:34.274571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.274280    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4" event={"ID":"ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd","Type":"ContainerDied","Data":"30ef31dc9664cf57e76c3ce160a85352fdbd5b3ef0b5b6ca1fd9347fc1bb56e3"}
Nov 03 22:52:34.274571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.274302    5647 scope.go:122] "RemoveContainer" containerID="f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346"
Nov 03 22:52:34.285277 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.285257    5647 scope.go:122] "RemoveContainer" containerID="f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346"
Nov 03 22:52:34.285568 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:52:34.285546    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346\": container with ID starting with f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346 not found: ID does not exist" containerID="f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346"
Nov 03 22:52:34.285619 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.285580    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346"} err="failed to get container status \"f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346\": rpc error: code = NotFound desc = could not find container \"f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346\": container with ID starting with f8943d4e6a2b8891277e3c3eb754331b98da078cdcef9015fba01820ce96a346 not found: ID does not exist"
Nov 03 22:52:34.441024 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.440996    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"]
Nov 03 22:52:34.491001 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.490969    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4"]
Nov 03 22:52:34.720758 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:34.719551    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd" path="/var/lib/kubelet/pods/ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd/volumes"
Nov 03 22:52:41.796636 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.769524    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"]
Nov 03 22:52:41.886491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.886455    5647 cpu_manager.go:414] "containerMap: removing container" podUID="ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd" containerName="payload-processing"
Nov 03 22:52:41.886775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.886755    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd" containerName="payload-processing"
Nov 03 22:52:41.887109 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.887092    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad2f6eec-aab0-4ea6-83e8-5ea05cd62afd" containerName="payload-processing"
Nov 03 22:52:41.911057 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.911008    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"]
Nov 03 22:52:41.911261 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.911150    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:41.988866 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.965024    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-26444f\""
Nov 03 22:52:42.004724 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.990792    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx9k5\" (UniqueName: \"kubernetes.io/projected/9f64e34b-2d3a-4298-b410-40d4d7a01334-kube-api-access-zx9k5\") pod \"payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks\" (UID: \"9f64e34b-2d3a-4298-b410-40d4d7a01334\") " pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:42.004724 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:41.991005    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f64e34b-2d3a-4298-b410-40d4d7a01334-plugins-config-volume\") pod \"payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks\" (UID: \"9f64e34b-2d3a-4298-b410-40d4d7a01334\") " pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:42.049779 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:42.045454    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-26444f-dockercfg-59m5j\""
Nov 03 22:52:42.092771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:42.092724    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx9k5\" (UniqueName: \"kubernetes.io/projected/9f64e34b-2d3a-4298-b410-40d4d7a01334-kube-api-access-zx9k5\") pod \"payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks\" (UID: \"9f64e34b-2d3a-4298-b410-40d4d7a01334\") " pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:42.092962 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:42.092884    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f64e34b-2d3a-4298-b410-40d4d7a01334-plugins-config-volume\") pod \"payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks\" (UID: \"9f64e34b-2d3a-4298-b410-40d4d7a01334\") " pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:42.093929 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:42.093901    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f64e34b-2d3a-4298-b410-40d4d7a01334-plugins-config-volume\") pod \"payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks\" (UID: \"9f64e34b-2d3a-4298-b410-40d4d7a01334\") " pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:42.153567 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:42.153532    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx9k5\" (UniqueName: \"kubernetes.io/projected/9f64e34b-2d3a-4298-b410-40d4d7a01334-kube-api-access-zx9k5\") pod \"payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks\" (UID: \"9f64e34b-2d3a-4298-b410-40d4d7a01334\") " pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:42.331225 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:42.331142    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:42.717204 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:52:42.717167    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f64e34b_2d3a_4298_b410_40d4d7a01334.slice/crio-9b1ce8a1d1e65b3d6002549b01eba91c3033b5d24e9de615d6fa89e8f69d5f26 WatchSource:0}: Error finding container 9b1ce8a1d1e65b3d6002549b01eba91c3033b5d24e9de615d6fa89e8f69d5f26: Status 404 returned error can't find the container with id 9b1ce8a1d1e65b3d6002549b01eba91c3033b5d24e9de615d6fa89e8f69d5f26
Nov 03 22:52:42.719238 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:42.719210    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"]
Nov 03 22:52:43.416654 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:43.416612    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks" event={"ID":"9f64e34b-2d3a-4298-b410-40d4d7a01334","Type":"ContainerStarted","Data":"af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4"}
Nov 03 22:52:43.417172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:43.416775    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:43.417172 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:43.416812    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks" event={"ID":"9f64e34b-2d3a-4298-b410-40d4d7a01334","Type":"ContainerStarted","Data":"9b1ce8a1d1e65b3d6002549b01eba91c3033b5d24e9de615d6fa89e8f69d5f26"}
Nov 03 22:52:43.534530 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:43.534464    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks" podStartSLOduration=2.534444219 podStartE2EDuration="2.534444219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:52:43.531334923 +0000 UTC m=+1786.971872182" watchObservedRunningTime="2026-11-03 22:52:43.534444219 +0000 UTC m=+1786.974981480"
Nov 03 22:52:45.724238 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:45.723636    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": read tcp 10.134.0.2:43426->10.134.0.111:15021: read: connection reset by peer" start-of-body=
Nov 03 22:52:45.724238 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:45.723712    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": read tcp 10.134.0.2:43426->10.134.0.111:15021: read: connection reset by peer"
Nov 03 22:52:46.044235 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:52:46.044147    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:52:46.500975 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:46.500941    5647 generic.go:356] "Generic (PLEG): container finished" podID="63034fce-49e9-40a2-ae8b-818725c20288" containerID="78d24639b4b8ad9fb47fdadb4300e81f61883b10b802a195d58a4e62e65edf01" exitCode=0
Nov 03 22:52:46.501178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:46.501010    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" event={"ID":"63034fce-49e9-40a2-ae8b-818725c20288","Type":"ContainerDied","Data":"78d24639b4b8ad9fb47fdadb4300e81f61883b10b802a195d58a4e62e65edf01"}
Nov 03 22:52:46.501496 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:46.501478    5647 scope.go:122] "RemoveContainer" containerID="78d24639b4b8ad9fb47fdadb4300e81f61883b10b802a195d58a4e62e65edf01"
Nov 03 22:52:46.503292 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:46.503264    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:52:46.503356 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:46.503331    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:52:46.503400 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:46.503361    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:52:47.551313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:47.551270    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" event={"ID":"63034fce-49e9-40a2-ae8b-818725c20288","Type":"ContainerStarted","Data":"8e81906d0a8740af759617d696612b163effd5625249146918f429599adc9afb"}
Nov 03 22:52:48.470726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:48.468039    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:52:48.470726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:48.468176    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 22:52:48.470726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:48.468233    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 22:52:49.461358 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:49.461324    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 22:52:49.461788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:49.461386    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 22:52:50.460823 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:50.460783    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 22:52:50.461000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:50.460841    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 22:52:51.461603 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:51.461567    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 22:52:51.462044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:51.461626    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 22:52:52.461689 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:52.461082    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Nov 03 22:52:52.461689 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:52.461146    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Nov 03 22:52:53.465016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:53.464984    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:52:53.465415 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:53.465384    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:52:53.466161 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:53.466143    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:52:54.426888 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:54.426857    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:52:56.964706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:56.964652    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:52:56.970615 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:52:56.970594    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:53:08.065953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.065762    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"]
Nov 03 22:53:08.087525 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.087493    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"]
Nov 03 22:53:08.087702 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.087599    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:08.139579 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.139555    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-0406a8\""
Nov 03 22:53:08.188957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.188929    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-0406a8-dockercfg-cf699\""
Nov 03 22:53:08.218838 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.218800    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks2t2\" (UniqueName: \"kubernetes.io/projected/b63a92a7-bd36-435b-b0af-1967f048af8b-kube-api-access-ks2t2\") pod \"payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr\" (UID: \"b63a92a7-bd36-435b-b0af-1967f048af8b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:08.218982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.218875    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b63a92a7-bd36-435b-b0af-1967f048af8b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr\" (UID: \"b63a92a7-bd36-435b-b0af-1967f048af8b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:08.321065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.320987    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks2t2\" (UniqueName: \"kubernetes.io/projected/b63a92a7-bd36-435b-b0af-1967f048af8b-kube-api-access-ks2t2\") pod \"payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr\" (UID: \"b63a92a7-bd36-435b-b0af-1967f048af8b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:08.321200 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.321080    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b63a92a7-bd36-435b-b0af-1967f048af8b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr\" (UID: \"b63a92a7-bd36-435b-b0af-1967f048af8b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:08.321858 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.321834    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b63a92a7-bd36-435b-b0af-1967f048af8b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr\" (UID: \"b63a92a7-bd36-435b-b0af-1967f048af8b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:08.381890 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.381854    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks2t2\" (UniqueName: \"kubernetes.io/projected/b63a92a7-bd36-435b-b0af-1967f048af8b-kube-api-access-ks2t2\") pod \"payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr\" (UID: \"b63a92a7-bd36-435b-b0af-1967f048af8b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:08.401351 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.401318    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:08.790590 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:08.786353    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"]
Nov 03 22:53:09.091737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:09.091624    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr" event={"ID":"b63a92a7-bd36-435b-b0af-1967f048af8b","Type":"ContainerStarted","Data":"43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477"}
Nov 03 22:53:09.091737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:09.091710    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:09.091737 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:09.091731    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr" event={"ID":"b63a92a7-bd36-435b-b0af-1967f048af8b","Type":"ContainerStarted","Data":"86c98660032065117521af0367c2dc029e55de82623328cad78b0d220bbb8d46"}
Nov 03 22:53:09.208469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:09.208406    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr" podStartSLOduration=2.208388512 podStartE2EDuration="2.208388512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:53:09.205928484 +0000 UTC m=+1812.646465744" watchObservedRunningTime="2026-11-03 22:53:09.208388512 +0000 UTC m=+1812.648925795"
Nov 03 22:53:16.603692 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:53:16.601012    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:53:20.101870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:20.101840    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:53:21.758465 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:21.758420    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"]
Nov 03 22:53:21.758976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:21.758741    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" containerID="cri-o://41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b" gracePeriod=30
Nov 03 22:53:25.225487 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.225448    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"]
Nov 03 22:53:25.230507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.230483    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:25.314648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.314603    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"]
Nov 03 22:53:25.349055 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.349016    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w4-aacab9\""
Nov 03 22:53:25.358702 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.357420    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"]
Nov 03 22:53:25.369218 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.369173    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w4-aacab9-dockercfg-x8ff6\""
Nov 03 22:53:25.371605 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.371582    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:25.429731 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.428180    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6fdj\" (UniqueName: \"kubernetes.io/projected/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-kube-api-access-g6fdj\") pod \"payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn\" (UID: \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:25.429731 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.428427    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f615715-b0f7-4754-b079-f998dd0cf49d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq\" (UID: \"9f615715-b0f7-4754-b079-f998dd0cf49d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:25.429731 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.428588    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twqhd\" (UniqueName: \"kubernetes.io/projected/9f615715-b0f7-4754-b079-f998dd0cf49d-kube-api-access-twqhd\") pod \"payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq\" (UID: \"9f615715-b0f7-4754-b079-f998dd0cf49d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:25.429731 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.428745    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn\" (UID: \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:25.721464 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.658377    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6fdj\" (UniqueName: \"kubernetes.io/projected/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-kube-api-access-g6fdj\") pod \"payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn\" (UID: \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:25.722015 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.721985    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f615715-b0f7-4754-b079-f998dd0cf49d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq\" (UID: \"9f615715-b0f7-4754-b079-f998dd0cf49d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:25.722365 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.722348    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twqhd\" (UniqueName: \"kubernetes.io/projected/9f615715-b0f7-4754-b079-f998dd0cf49d-kube-api-access-twqhd\") pod \"payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq\" (UID: \"9f615715-b0f7-4754-b079-f998dd0cf49d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:25.723216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.723192    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn\" (UID: \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:25.726108 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.726084    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn\" (UID: \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:25.727914 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.726466    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f615715-b0f7-4754-b079-f998dd0cf49d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq\" (UID: \"9f615715-b0f7-4754-b079-f998dd0cf49d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:25.728460 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.728367    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"]
Nov 03 22:53:25.734361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.732829    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6fdj\" (UniqueName: \"kubernetes.io/projected/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-kube-api-access-g6fdj\") pod \"payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn\" (UID: \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:25.789291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.789258    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twqhd\" (UniqueName: \"kubernetes.io/projected/9f615715-b0f7-4754-b079-f998dd0cf49d-kube-api-access-twqhd\") pod \"payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq\" (UID: \"9f615715-b0f7-4754-b079-f998dd0cf49d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:25.812245 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.812214    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:25.848510 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:25.848477    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:26.212286 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.212250    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"]
Nov 03 22:53:26.253927 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:53:26.253898    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f615715_b0f7_4754_b079_f998dd0cf49d.slice/crio-2d245c7b9a1aa89cf9a944d162535081d8b611e9efdf516fdc231c75901ff9c3 WatchSource:0}: Error finding container 2d245c7b9a1aa89cf9a944d162535081d8b611e9efdf516fdc231c75901ff9c3: Status 404 returned error can't find the container with id 2d245c7b9a1aa89cf9a944d162535081d8b611e9efdf516fdc231c75901ff9c3
Nov 03 22:53:26.270218 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.270173    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"]
Nov 03 22:53:26.270612 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:53:26.270591    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1399892_8ce0_410d_9bd8_d1a2a40df2e8.slice/crio-75d238751252a0215f5df7862b957d8c3a3a627b93c221333f4ffbd88133be15 WatchSource:0}: Error finding container 75d238751252a0215f5df7862b957d8c3a3a627b93c221333f4ffbd88133be15: Status 404 returned error can't find the container with id 75d238751252a0215f5df7862b957d8c3a3a627b93c221333f4ffbd88133be15
Nov 03 22:53:26.560364 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.560264    5647 patch_prober.go:29] interesting pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:53:26.560512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.560347    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:53:26.746159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.746120    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq" event={"ID":"9f615715-b0f7-4754-b079-f998dd0cf49d","Type":"ContainerStarted","Data":"5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f"}
Nov 03 22:53:26.746338 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.746179    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq" event={"ID":"9f615715-b0f7-4754-b079-f998dd0cf49d","Type":"ContainerStarted","Data":"2d245c7b9a1aa89cf9a944d162535081d8b611e9efdf516fdc231c75901ff9c3"}
Nov 03 22:53:26.746338 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.746232    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:26.748206 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.748183    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn" event={"ID":"c1399892-8ce0-410d-9bd8-d1a2a40df2e8","Type":"ContainerStarted","Data":"a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846"}
Nov 03 22:53:26.748314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.748221    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn" event={"ID":"c1399892-8ce0-410d-9bd8-d1a2a40df2e8","Type":"ContainerStarted","Data":"75d238751252a0215f5df7862b957d8c3a3a627b93c221333f4ffbd88133be15"}
Nov 03 22:53:26.748314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.748271    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:26.862134 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.862030    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq" podStartSLOduration=1.8620176819999998 podStartE2EDuration="1.862017682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:53:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:53:26.860441123 +0000 UTC m=+1830.300978382" watchObservedRunningTime="2026-11-03 22:53:26.862017682 +0000 UTC m=+1830.302554941"
Nov 03 22:53:26.994761 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:26.991919    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn" podStartSLOduration=1.991897606 podStartE2EDuration="1.991897606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:53:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:53:26.975229802 +0000 UTC m=+1830.415767058" watchObservedRunningTime="2026-11-03 22:53:26.991897606 +0000 UTC m=+1830.432434866"
Nov 03 22:53:27.172430 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.172385    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 22:53:27.172577 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.172473    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:53:27.224371 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.224344    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:53:27.306999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.306967    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-kube-api-access-488h9\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-kube-api-access-488h9\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.307475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307032    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-credential-socket\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.307475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307155    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-certs\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.307475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307239    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-istio-token\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-istio-token\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.307475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307275    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f4d940d-10b6-41f2-a286-937c804985c4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4d940d-10b6-41f2-a286-937c804985c4-istiod-ca-cert\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.307475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307351    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-envoy\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.307825 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307508    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-data\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.307825 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307570    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-certs" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:53:27.307825 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307577    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-socket\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.307825 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307707    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6f4d940d-10b6-41f2-a286-937c804985c4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4d940d-10b6-41f2-a286-937c804985c4-istio-podinfo\") pod \"6f4d940d-10b6-41f2-a286-937c804985c4\" (UID: \"6f4d940d-10b6-41f2-a286-937c804985c4\") "
Nov 03 22:53:27.308045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307847    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-credential-socket" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:53:27.308045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.307878    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-socket" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:53:27.308160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.308078    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-data" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:53:27.308349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.308324    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f4d940d-10b6-41f2-a286-937c804985c4-istiod-ca-cert" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:53:27.308541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.308520    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.308648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.308546    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f4d940d-10b6-41f2-a286-937c804985c4-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.308648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.308562    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.308648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.308575    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.308648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.308588    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.310926 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.310899    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-envoy" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:53:27.311133 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.311108    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-kube-api-access-488h9" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "kube-api-access-488h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:53:27.317097 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.317068    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f4d940d-10b6-41f2-a286-937c804985c4-istio-podinfo" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:53:27.317400 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.317380    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-istio-token" pod "6f4d940d-10b6-41f2-a286-937c804985c4" (UID: "6f4d940d-10b6-41f2-a286-937c804985c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:53:27.410217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.410186    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.410217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.410215    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f4d940d-10b6-41f2-a286-937c804985c4-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.410217 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.410224    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f4d940d-10b6-41f2-a286-937c804985c4-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.410450 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.410234    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-488h9\" (UniqueName: \"kubernetes.io/projected/6f4d940d-10b6-41f2-a286-937c804985c4-kube-api-access-488h9\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:53:27.757110 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.757071    5647 generic.go:356] "Generic (PLEG): container finished" podID="6f4d940d-10b6-41f2-a286-937c804985c4" containerID="41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b" exitCode=0
Nov 03 22:53:27.757314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.757171    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"
Nov 03 22:53:27.757314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.757211    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" event={"ID":"6f4d940d-10b6-41f2-a286-937c804985c4","Type":"ContainerDied","Data":"41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b"}
Nov 03 22:53:27.757314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.757273    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb" event={"ID":"6f4d940d-10b6-41f2-a286-937c804985c4","Type":"ContainerDied","Data":"4a2d1a6e41e1151da244550c190892beb7c2534e9307b0cdf706a9997968aaf3"}
Nov 03 22:53:27.757480 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.757303    5647 scope.go:122] "RemoveContainer" containerID="41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b"
Nov 03 22:53:27.769224 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.769202    5647 scope.go:122] "RemoveContainer" containerID="41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b"
Nov 03 22:53:27.769537 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:53:27.769512    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b\": container with ID starting with 41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b not found: ID does not exist" containerID="41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b"
Nov 03 22:53:27.769594 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.769552    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b"} err="failed to get container status \"41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b\": rpc error: code = NotFound desc = could not find container \"41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b\": container with ID starting with 41bb66f6d8455dfb60bedf9bd5df7ae05d5075bb0994a61843e8737e1442bf1b not found: ID does not exist"
Nov 03 22:53:27.922617 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.922585    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"]
Nov 03 22:53:27.971822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:27.971788    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb"]
Nov 03 22:53:28.766216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:28.765588    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" path="/var/lib/kubelet/pods/6f4d940d-10b6-41f2-a286-937c804985c4/volumes"
Nov 03 22:53:28.782038 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:28.782008    5647 generic.go:356] "Generic (PLEG): container finished" podID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerID="ed3507663c10e41044028ea6347525817e38e35ae49788fbca0fe06741a978b8" exitCode=0
Nov 03 22:53:28.782219 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:28.782072    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" event={"ID":"ea7424a3-4b15-44af-beae-c41b3eb13801","Type":"ContainerDied","Data":"ed3507663c10e41044028ea6347525817e38e35ae49788fbca0fe06741a978b8"}
Nov 03 22:53:28.782539 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:28.782522    5647 scope.go:122] "RemoveContainer" containerID="ed3507663c10e41044028ea6347525817e38e35ae49788fbca0fe06741a978b8"
Nov 03 22:53:28.784342 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:28.784318    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:53:28.784411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:28.784380    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:53:28.784455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:28.784411    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:53:29.804066 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:29.804030    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" event={"ID":"ea7424a3-4b15-44af-beae-c41b3eb13801","Type":"ContainerStarted","Data":"ffb16ba850205ea68fb7cf264d7082b0be13104d55ebbac32685cce5e03c7ea6"}
Nov 03 22:53:30.436571 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:30.436538    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:53:30.436817 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:30.436719    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 22:53:30.436817 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:30.436781    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 22:53:31.436995 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:31.436960    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 22:53:31.437518 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:31.437023    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 22:53:32.468369 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:32.468330    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 22:53:32.469033 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:32.468998    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 22:53:33.437248 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:33.437205    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 22:53:33.437434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:33.437283    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 22:53:34.436892 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:34.436851    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 22:53:34.437396 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:34.436931    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 22:53:35.437059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:35.437019    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Nov 03 22:53:35.437486 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:35.437125    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Nov 03 22:53:36.442784 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:36.442755    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:53:36.443243 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:36.443162    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:53:36.443954 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:36.443938    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 22:53:37.760711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:37.760644    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:53:37.761165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:37.760824    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:53:46.055778 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:53:46.055741    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:53:58.391820 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:53:58.391789    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:54:09.558173 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:09.558136    5647 generic.go:356] "Generic (PLEG): container finished" podID="928babb1-e87e-4ce6-b257-d149c947c724" containerID="e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39" exitCode=0
Nov 03 22:54:09.558605 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:09.558208    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" event={"ID":"928babb1-e87e-4ce6-b257-d149c947c724","Type":"ContainerDied","Data":"e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39"}
Nov 03 22:54:09.558722 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:09.558708    5647 scope.go:122] "RemoveContainer" containerID="e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39"
Nov 03 22:54:09.560470 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:09.560443    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:54:09.560548 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:09.560510    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:54:09.560548 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:09.560542    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:54:10.569264 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:10.569221    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" event={"ID":"928babb1-e87e-4ce6-b257-d149c947c724","Type":"ContainerStarted","Data":"f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f"}
Nov 03 22:54:11.250165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:11.250133    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:54:11.250367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:11.250262    5647 patch_prober.go:29] interesting pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Nov 03 22:54:11.250367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:11.250325    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Nov 03 22:54:12.250595 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:12.250560    5647 patch_prober.go:29] interesting pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Nov 03 22:54:12.251045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:12.250608    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Nov 03 22:54:13.250244 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:13.250210    5647 patch_prober.go:29] interesting pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Nov 03 22:54:13.250429 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:13.250261    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Nov 03 22:54:14.249802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:14.249768    5647 patch_prober.go:29] interesting pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Nov 03 22:54:14.250224 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:14.249819    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Nov 03 22:54:15.250100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:15.250060    5647 patch_prober.go:29] interesting pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Nov 03 22:54:15.250515 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:15.250132    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Nov 03 22:54:16.169715 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:54:16.165143    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:54:16.277696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:16.270019    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:54:16.277696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:16.271799    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:54:16.295696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:16.290658    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:54:23.306637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:23.306598    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"]
Nov 03 22:54:23.307056 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:23.306828    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy" containerID="cri-o://e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37" gracePeriod=30
Nov 03 22:54:25.435373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:25.435340    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:54:25.435831 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:25.435396    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:54:28.584092 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.584030    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:54:28.665042 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665008    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-data\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665219 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665101    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-certs\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665219 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665146    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0753c62f-7777-49c6-8090-bc2093554768-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0753c62f-7777-49c6-8090-bc2093554768-istiod-ca-cert\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665219 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665206    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-data" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:28.665366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665230    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-kube-api-access-qkhd7\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-kube-api-access-qkhd7\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665267    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-istio-token\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-istio-token\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665295    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-socket\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665373    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-certs" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:28.665544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665440    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-credential-socket\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665472    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-envoy\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665553    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0753c62f-7777-49c6-8090-bc2093554768-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0753c62f-7777-49c6-8090-bc2093554768-istio-podinfo\") pod \"0753c62f-7777-49c6-8090-bc2093554768\" (UID: \"0753c62f-7777-49c6-8090-bc2093554768\") "
Nov 03 22:54:28.665747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665561    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-socket" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:28.665747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665582    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0753c62f-7777-49c6-8090-bc2093554768-istiod-ca-cert" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:54:28.665747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.665732    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-credential-socket" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:28.666384 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.666358    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.666499 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.666387    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0753c62f-7777-49c6-8090-bc2093554768-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.666499 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.666404    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.666499 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.666419    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.666499 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.666433    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.667725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.667652    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-istio-token" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:28.667725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.667703    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-kube-api-access-qkhd7" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "kube-api-access-qkhd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:28.667725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.667702    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-envoy" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:54:28.667898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.667849    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0753c62f-7777-49c6-8090-bc2093554768-istio-podinfo" pod "0753c62f-7777-49c6-8090-bc2093554768" (UID: "0753c62f-7777-49c6-8090-bc2093554768"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:54:28.746660 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.746625    5647 generic.go:356] "Generic (PLEG): container finished" podID="0753c62f-7777-49c6-8090-bc2093554768" containerID="e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37" exitCode=0
Nov 03 22:54:28.746859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.746689    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"
Nov 03 22:54:28.746859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.746698    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" event={"ID":"0753c62f-7777-49c6-8090-bc2093554768","Type":"ContainerDied","Data":"e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37"}
Nov 03 22:54:28.746859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.746740    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq" event={"ID":"0753c62f-7777-49c6-8090-bc2093554768","Type":"ContainerDied","Data":"4e7050959b81556b8c535c969ba5c4079aed42a507e1255dc6727ada512402eb"}
Nov 03 22:54:28.746859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.746765    5647 scope.go:122] "RemoveContainer" containerID="e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37"
Nov 03 22:54:28.758812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.758712    5647 scope.go:122] "RemoveContainer" containerID="e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37"
Nov 03 22:54:28.759361 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:54:28.759326    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37\": container with ID starting with e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37 not found: ID does not exist" containerID="e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37"
Nov 03 22:54:28.759479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.759367    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37"} err="failed to get container status \"e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37\": rpc error: code = NotFound desc = could not find container \"e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37\": container with ID starting with e3cb4b79c0e613f7ab06813af2b6bf44004a9934ef2766667405d82ada66fd37 not found: ID does not exist"
Nov 03 22:54:28.768960 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.768926    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkhd7\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-kube-api-access-qkhd7\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.768960 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.768963    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0753c62f-7777-49c6-8090-bc2093554768-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.769202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.768979    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0753c62f-7777-49c6-8090-bc2093554768-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.769202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.768994    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0753c62f-7777-49c6-8090-bc2093554768-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:28.877568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.877442    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"]
Nov 03 22:54:28.881846 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:28.881807    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq"]
Nov 03 22:54:30.698701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:30.698649    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0753c62f-7777-49c6-8090-bc2093554768" path="/var/lib/kubelet/pods/0753c62f-7777-49c6-8090-bc2093554768/volumes"
Nov 03 22:54:31.715159 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:31.715130    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"]
Nov 03 22:54:31.820291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:31.820259    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"]
Nov 03 22:54:32.264588 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.264553    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"]
Nov 03 22:54:32.265901 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.265876    5647 cpu_manager.go:414] "containerMap: removing container" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy"
Nov 03 22:54:32.265971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.265907    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy"
Nov 03 22:54:32.266008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.265974    5647 cpu_manager.go:414] "containerMap: removing container" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy"
Nov 03 22:54:32.266008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.265984    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy"
Nov 03 22:54:32.266167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.266155    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f4d940d-10b6-41f2-a286-937c804985c4" containerName="istio-proxy"
Nov 03 22:54:32.266199 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.266185    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="0753c62f-7777-49c6-8090-bc2093554768" containerName="istio-proxy"
Nov 03 22:54:32.270223 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.270204    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.315372 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.315338    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"]
Nov 03 22:54:32.322173 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.322138    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-0406a8-serving-cert\""
Nov 03 22:54:32.427320 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.427283    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fslh7\" (UniqueName: \"kubernetes.io/projected/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-kube-api-access-fslh7\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.427523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.427378    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.427523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.427432    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.427523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.427493    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.528741 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.528612    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.528910 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:54:32.528856    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-serving-cert: secret "maas-api-e2e-shared-b-w0-0406a8-serving-cert" not found
Nov 03 22:54:32.528910 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.528876    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fslh7\" (UniqueName: \"kubernetes.io/projected/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-kube-api-access-fslh7\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.529051 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.528919    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.529051 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:54:32.528949    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls podName:fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833 nodeName:}" failed. No retries permitted until 2026-11-03 22:54:33.028927007 +0000 UTC m=+1896.469464266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls") pod "maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx" (UID: "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833") : secret "maas-api-e2e-shared-b-w0-0406a8-serving-cert" not found
Nov 03 22:54:32.529051 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.529030    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.529549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.529524    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.532300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.532273    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:32.642402 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:32.636832    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fslh7\" (UniqueName: \"kubernetes.io/projected/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-kube-api-access-fslh7\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:33.047151 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:33.047117    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:33.049698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:33.049651    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:33.183486 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:33.183439    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:34.085111 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:34.085081    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"]
Nov 03 22:54:34.086067 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:54:34.086043    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc1f1d1_8fb2_4e15_853f_2bbad5fa9833.slice/crio-be5b48ed79bebe4efa957353eced4341d40331c57d06e0933d4336d80663f75d WatchSource:0}: Error finding container be5b48ed79bebe4efa957353eced4341d40331c57d06e0933d4336d80663f75d: Status 404 returned error can't find the container with id be5b48ed79bebe4efa957353eced4341d40331c57d06e0933d4336d80663f75d
Nov 03 22:54:34.839325 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:34.839287    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx" event={"ID":"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833","Type":"ContainerStarted","Data":"be5b48ed79bebe4efa957353eced4341d40331c57d06e0933d4336d80663f75d"}
Nov 03 22:54:35.852759 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:35.852705    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx" event={"ID":"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833","Type":"ContainerStarted","Data":"b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d"}
Nov 03 22:54:35.853320 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:35.853290    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:35.972949 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:35.972896    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx" podStartSLOduration=3.165756456 podStartE2EDuration="3.972881216s" totalImagesPullingTime="807.12476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:54:35.969204258 +0000 UTC m=+1899.409741520" watchObservedRunningTime="2026-11-03 22:54:35.972881216 +0000 UTC m=+1899.413418475"
Nov 03 22:54:36.715479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:36.715398    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs" podUID="97183b8c-48d7-4950-9a67-327c011f0af6" containerName="payload-processing" containerID="cri-o://167e23dc99d7aa330b8ddfb840f016bbf60af5c413f58dc7ea5115367c7e7f9a" gracePeriod=25
Nov 03 22:54:36.820579 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:36.820539    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb" podUID="1fb2147c-d3fa-491e-be7b-e4be0b669502" containerName="payload-pre-processing" containerID="cri-o://116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35" gracePeriod=25
Nov 03 22:54:36.861613 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:36.861583    5647 generic.go:356] "Generic (PLEG): container finished" podID="97183b8c-48d7-4950-9a67-327c011f0af6" containerID="167e23dc99d7aa330b8ddfb840f016bbf60af5c413f58dc7ea5115367c7e7f9a" exitCode=0
Nov 03 22:54:36.862056 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:36.861663    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs" event={"ID":"97183b8c-48d7-4950-9a67-327c011f0af6","Type":"ContainerDied","Data":"167e23dc99d7aa330b8ddfb840f016bbf60af5c413f58dc7ea5115367c7e7f9a"}
Nov 03 22:54:37.066236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.066207    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:54:37.179394 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.179366    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:54:37.180829 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.180805    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97183b8c-48d7-4950-9a67-327c011f0af6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97183b8c-48d7-4950-9a67-327c011f0af6-plugins-config-volume\") pod \"97183b8c-48d7-4950-9a67-327c011f0af6\" (UID: \"97183b8c-48d7-4950-9a67-327c011f0af6\") "
Nov 03 22:54:37.181033 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.181020    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97183b8c-48d7-4950-9a67-327c011f0af6-kube-api-access-cx8h7\" (UniqueName: \"kubernetes.io/projected/97183b8c-48d7-4950-9a67-327c011f0af6-kube-api-access-cx8h7\") pod \"97183b8c-48d7-4950-9a67-327c011f0af6\" (UID: \"97183b8c-48d7-4950-9a67-327c011f0af6\") "
Nov 03 22:54:37.181126 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.181104    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97183b8c-48d7-4950-9a67-327c011f0af6-plugins-config-volume" pod "97183b8c-48d7-4950-9a67-327c011f0af6" (UID: "97183b8c-48d7-4950-9a67-327c011f0af6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:54:37.181605 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.181575    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97183b8c-48d7-4950-9a67-327c011f0af6-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:37.183735 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.183711    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97183b8c-48d7-4950-9a67-327c011f0af6-kube-api-access-cx8h7" pod "97183b8c-48d7-4950-9a67-327c011f0af6" (UID: "97183b8c-48d7-4950-9a67-327c011f0af6"). InnerVolumeSpecName "kube-api-access-cx8h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:37.283175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.283097    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fb2147c-d3fa-491e-be7b-e4be0b669502-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fb2147c-d3fa-491e-be7b-e4be0b669502-plugins-config-volume\") pod \"1fb2147c-d3fa-491e-be7b-e4be0b669502\" (UID: \"1fb2147c-d3fa-491e-be7b-e4be0b669502\") "
Nov 03 22:54:37.283175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.283144    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fb2147c-d3fa-491e-be7b-e4be0b669502-kube-api-access-4gmhj\" (UniqueName: \"kubernetes.io/projected/1fb2147c-d3fa-491e-be7b-e4be0b669502-kube-api-access-4gmhj\") pod \"1fb2147c-d3fa-491e-be7b-e4be0b669502\" (UID: \"1fb2147c-d3fa-491e-be7b-e4be0b669502\") "
Nov 03 22:54:37.283464 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.283439    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fb2147c-d3fa-491e-be7b-e4be0b669502-plugins-config-volume" pod "1fb2147c-d3fa-491e-be7b-e4be0b669502" (UID: "1fb2147c-d3fa-491e-be7b-e4be0b669502"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:54:37.283859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.283836    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx8h7\" (UniqueName: \"kubernetes.io/projected/97183b8c-48d7-4950-9a67-327c011f0af6-kube-api-access-cx8h7\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:37.283859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.283858    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fb2147c-d3fa-491e-be7b-e4be0b669502-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:37.285591 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.285566    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fb2147c-d3fa-491e-be7b-e4be0b669502-kube-api-access-4gmhj" pod "1fb2147c-d3fa-491e-be7b-e4be0b669502" (UID: "1fb2147c-d3fa-491e-be7b-e4be0b669502"). InnerVolumeSpecName "kube-api-access-4gmhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:54:37.385857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.385805    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gmhj\" (UniqueName: \"kubernetes.io/projected/1fb2147c-d3fa-491e-be7b-e4be0b669502-kube-api-access-4gmhj\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:54:37.870183 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.870152    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"
Nov 03 22:54:37.870634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.870183    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs" event={"ID":"97183b8c-48d7-4950-9a67-327c011f0af6","Type":"ContainerDied","Data":"94543019c0b3b5f3b62f3a8aba015e6abcc707af2d51bcf8571d04db5fcf7b8e"}
Nov 03 22:54:37.870634 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.870257    5647 scope.go:122] "RemoveContainer" containerID="167e23dc99d7aa330b8ddfb840f016bbf60af5c413f58dc7ea5115367c7e7f9a"
Nov 03 22:54:37.871647 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.871578    5647 generic.go:356] "Generic (PLEG): container finished" podID="1fb2147c-d3fa-491e-be7b-e4be0b669502" containerID="116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35" exitCode=0
Nov 03 22:54:37.871647 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.871618    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"
Nov 03 22:54:37.871810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.871657    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb" event={"ID":"1fb2147c-d3fa-491e-be7b-e4be0b669502","Type":"ContainerDied","Data":"116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35"}
Nov 03 22:54:37.871810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.871743    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb" event={"ID":"1fb2147c-d3fa-491e-be7b-e4be0b669502","Type":"ContainerDied","Data":"9895d6cc6287f0418b59fa455d29ccf9f30678825a7afc33eaa6087d97a7b5a0"}
Nov 03 22:54:37.881301 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.881202    5647 scope.go:122] "RemoveContainer" containerID="116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35"
Nov 03 22:54:37.890823 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.890803    5647 scope.go:122] "RemoveContainer" containerID="116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35"
Nov 03 22:54:37.891111 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:54:37.891089    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35\": container with ID starting with 116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35 not found: ID does not exist" containerID="116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35"
Nov 03 22:54:37.891223 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:37.891117    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35"} err="failed to get container status \"116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35\": rpc error: code = NotFound desc = could not find container \"116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35\": container with ID starting with 116fb4faa7d8f958c525aa0fb1db8358fda7bac95d07b5400e517bdcebd08c35 not found: ID does not exist"
Nov 03 22:54:38.044963 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:38.044873    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"]
Nov 03 22:54:38.085741 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:38.084942    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb"]
Nov 03 22:54:38.192946 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:38.192914    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"]
Nov 03 22:54:38.199829 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:38.199804    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs"]
Nov 03 22:54:38.697373 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:38.697342    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fb2147c-d3fa-491e-be7b-e4be0b669502" path="/var/lib/kubelet/pods/1fb2147c-d3fa-491e-be7b-e4be0b669502/volumes"
Nov 03 22:54:38.697949 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:38.697924    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97183b8c-48d7-4950-9a67-327c011f0af6" path="/var/lib/kubelet/pods/97183b8c-48d7-4950-9a67-327c011f0af6/volumes"
Nov 03 22:54:39.966045 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:39.966008    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:54:42.876772 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:54:42.876746    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:54:46.044680 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:54:46.044629    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:55:02.115966 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.115929    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"]
Nov 03 22:55:02.116921 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.116898    5647 cpu_manager.go:414] "containerMap: removing container" podUID="1fb2147c-d3fa-491e-be7b-e4be0b669502" containerName="payload-pre-processing"
Nov 03 22:55:02.117046 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.116921    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fb2147c-d3fa-491e-be7b-e4be0b669502" containerName="payload-pre-processing"
Nov 03 22:55:02.117046 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.116996    5647 cpu_manager.go:414] "containerMap: removing container" podUID="97183b8c-48d7-4950-9a67-327c011f0af6" containerName="payload-processing"
Nov 03 22:55:02.117046 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.117004    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97183b8c-48d7-4950-9a67-327c011f0af6" containerName="payload-processing"
Nov 03 22:55:02.117208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.117179    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fb2147c-d3fa-491e-be7b-e4be0b669502" containerName="payload-pre-processing"
Nov 03 22:55:02.117208 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.117205    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="97183b8c-48d7-4950-9a67-327c011f0af6" containerName="payload-processing"
Nov 03 22:55:02.122250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.122230    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.214581 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.214547    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-metrics-certs\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.214824 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.214802    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.215156 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.215137    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l99ln\" (UniqueName: \"kubernetes.io/projected/23e0576c-6a95-4420-9f3a-13524619f118-kube-api-access-l99ln\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.215211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.215187    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23e0576c-6a95-4420-9f3a-13524619f118-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.223637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.223603    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a91d2d-serving-cert\""
Nov 03 22:55:02.229720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.229695    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"]
Nov 03 22:55:02.316442 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.316407    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l99ln\" (UniqueName: \"kubernetes.io/projected/23e0576c-6a95-4420-9f3a-13524619f118-kube-api-access-l99ln\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.316625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.316455    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23e0576c-6a95-4420-9f3a-13524619f118-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.316625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.316496    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-metrics-certs\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.316625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.316590    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.316836 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:55:02.316814    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-serving-cert: secret "maas-api-e2e-inf-a-a91d2d-serving-cert" not found
Nov 03 22:55:02.316902 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:55:02.316891    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls podName:23e0576c-6a95-4420-9f3a-13524619f118 nodeName:}" failed. No retries permitted until 2026-11-03 22:55:02.816872738 +0000 UTC m=+1926.257409978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls") pod "maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s" (UID: "23e0576c-6a95-4420-9f3a-13524619f118") : secret "maas-api-e2e-inf-a-a91d2d-serving-cert" not found
Nov 03 22:55:02.317240 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.317215    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23e0576c-6a95-4420-9f3a-13524619f118-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.319276 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.319248    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-metrics-certs\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.419763 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.417928    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l99ln\" (UniqueName: \"kubernetes.io/projected/23e0576c-6a95-4420-9f3a-13524619f118-kube-api-access-l99ln\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.852778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.852715    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:02.855342 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:02.855317    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls\") pod \"maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:03.036153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:03.036119    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:03.420509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:03.420482    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"]
Nov 03 22:55:03.422030 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:55:03.422006    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e0576c_6a95_4420_9f3a_13524619f118.slice/crio-3e9cf2a492f3ec15df28b44add3c89fa6d660a52d3f0bf537a123888dc6b9340 WatchSource:0}: Error finding container 3e9cf2a492f3ec15df28b44add3c89fa6d660a52d3f0bf537a123888dc6b9340: Status 404 returned error can't find the container with id 3e9cf2a492f3ec15df28b44add3c89fa6d660a52d3f0bf537a123888dc6b9340
Nov 03 22:55:04.097743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:04.097698    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s" event={"ID":"23e0576c-6a95-4420-9f3a-13524619f118","Type":"ContainerStarted","Data":"3e9cf2a492f3ec15df28b44add3c89fa6d660a52d3f0bf537a123888dc6b9340"}
Nov 03 22:55:05.106623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:05.106582    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s" event={"ID":"23e0576c-6a95-4420-9f3a-13524619f118","Type":"ContainerStarted","Data":"823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334"}
Nov 03 22:55:05.107114 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:05.106699    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:11.120381 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:11.120347    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:55:11.239297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:11.239243    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s" podStartSLOduration=8.348205803 podStartE2EDuration="9.239230771s" totalImagesPullingTime="891.024968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:55:05.232957759 +0000 UTC m=+1928.673495019" watchObservedRunningTime="2026-11-03 22:55:11.239230771 +0000 UTC m=+1934.679768029"
Nov 03 22:55:16.044049 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:55:16.044001    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:55:24.390603 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:24.390561    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"]
Nov 03 22:55:24.391122 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:24.390819    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" containerID="cri-o://8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be" gracePeriod=30
Nov 03 22:55:25.227455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:25.227415    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:55:25.227618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:25.227471    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:55:28.540337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.540291    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"]
Nov 03 22:55:28.544965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.544942    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.650532    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.657881    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.657937    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fptpk\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-kube-api-access-fptpk\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.658018    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.658071    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.658162    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.658305    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a599efd3-eee5-41ff-9172-93bc1f287003-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.658407    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a599efd3-eee5-41ff-9172-93bc1f287003-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.658700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.658512    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.667137 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.667106    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"]
Nov 03 22:55:28.760334 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760283    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a599efd3-eee5-41ff-9172-93bc1f287003-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.760553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760389    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a599efd3-eee5-41ff-9172-93bc1f287003-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.760553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760456    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.760553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760476    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.760826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760569    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.760826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760603    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptpk\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-kube-api-access-fptpk\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.760826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760635    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.760826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760713    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.760826 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760775    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.761104 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.760980    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.761104 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.761044    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.761223 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.761198    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.761312 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.761290    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.761423 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.761297    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a599efd3-eee5-41ff-9172-93bc1f287003-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.763092 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.763071    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a599efd3-eee5-41ff-9172-93bc1f287003-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.763265 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.763247    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.821821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.821729    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.821988 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.821891    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fptpk\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-kube-api-access-fptpk\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qk9bp\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:28.859660 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:28.859630    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:29.295229 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.295198    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"]
Nov 03 22:55:29.295533 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:55:29.295512    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda599efd3_eee5_41ff_9172_93bc1f287003.slice/crio-606ecb16a9b496971975362819f487ccf46045c450e02b22608dda3fe251bc8e WatchSource:0}: Error finding container 606ecb16a9b496971975362819f487ccf46045c450e02b22608dda3fe251bc8e: Status 404 returned error can't find the container with id 606ecb16a9b496971975362819f487ccf46045c450e02b22608dda3fe251bc8e
Nov 03 22:55:29.298236 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.298206    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:55:29.298312 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.298271    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:55:29.298312 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.298299    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:55:29.307080 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.307051    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" event={"ID":"a599efd3-eee5-41ff-9172-93bc1f287003","Type":"ContainerStarted","Data":"606ecb16a9b496971975362819f487ccf46045c450e02b22608dda3fe251bc8e"}
Nov 03 22:55:29.649435 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.649407    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:55:29.778122 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778089    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-socket\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778148    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-certs\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778185    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-podinfo\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778305 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778221    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-data\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778394    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-token\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-token\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778424    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-credential-socket\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778490    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-envoy\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778488    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-certs" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:55:29.778800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778534    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-data" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:55:29.778800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778622    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-kube-api-access-zv8fj\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-kube-api-access-zv8fj\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778720    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a292b26-d09e-4f9b-8efd-53d8016b0475-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a292b26-d09e-4f9b-8efd-53d8016b0475-istiod-ca-cert\") pod \"5a292b26-d09e-4f9b-8efd-53d8016b0475\" (UID: \"5a292b26-d09e-4f9b-8efd-53d8016b0475\") "
Nov 03 22:55:29.778800 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.778735    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-credential-socket" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:55:29.779280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.779043    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-socket" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:55:29.779280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.779111    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a292b26-d09e-4f9b-8efd-53d8016b0475-istiod-ca-cert" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:55:29.780370 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.780056    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a292b26-d09e-4f9b-8efd-53d8016b0475-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:29.780370 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.780082    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:29.780370 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.780097    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:29.780370 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.780114    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:29.780370 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.780128    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:29.782041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.781908    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-podinfo" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:55:29.783897 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.783874    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-token" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:55:29.783976 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.783927    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-kube-api-access-zv8fj" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "kube-api-access-zv8fj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:55:29.784222 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.784205    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-envoy" pod "5a292b26-d09e-4f9b-8efd-53d8016b0475" (UID: "5a292b26-d09e-4f9b-8efd-53d8016b0475"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:55:29.894020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.893916    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:29.894020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.893956    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:29.894020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.893972    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv8fj\" (UniqueName: \"kubernetes.io/projected/5a292b26-d09e-4f9b-8efd-53d8016b0475-kube-api-access-zv8fj\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:29.894020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:29.893987    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a292b26-d09e-4f9b-8efd-53d8016b0475-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:30.315363 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.315332    5647 generic.go:356] "Generic (PLEG): container finished" podID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerID="8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be" exitCode=0
Nov 03 22:55:30.315568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.315445    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"
Nov 03 22:55:30.315568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.315478    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" event={"ID":"5a292b26-d09e-4f9b-8efd-53d8016b0475","Type":"ContainerDied","Data":"8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be"}
Nov 03 22:55:30.315568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.315540    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg" event={"ID":"5a292b26-d09e-4f9b-8efd-53d8016b0475","Type":"ContainerDied","Data":"bb06c03ac6746d0160bf0c49541acec2916ab64e92975a537d89e1f19fe4e59a"}
Nov 03 22:55:30.315787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.315575    5647 scope.go:122] "RemoveContainer" containerID="8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be"
Nov 03 22:55:30.317615 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.317587    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" event={"ID":"a599efd3-eee5-41ff-9172-93bc1f287003","Type":"ContainerStarted","Data":"e5bd53059e60c8c3bfac9a0a378e25f4aa9af00935d554ebfb1fe485645c464b"}
Nov 03 22:55:30.327410 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.327388    5647 scope.go:122] "RemoveContainer" containerID="8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be"
Nov 03 22:55:30.327734 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:55:30.327711    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be\": container with ID starting with 8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be not found: ID does not exist" containerID="8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be"
Nov 03 22:55:30.327835 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.327742    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be"} err="failed to get container status \"8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be\": rpc error: code = NotFound desc = could not find container \"8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be\": container with ID starting with 8d48d8cca3efd6f476ca66e2826d0eb76586bdedf50fe1cb89a1c7dfc6ce84be not found: ID does not exist"
Nov 03 22:55:30.497126 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.497014    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podStartSLOduration=2.496991635 podStartE2EDuration="2.496991635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:55:30.488261584 +0000 UTC m=+1953.928798877" watchObservedRunningTime="2026-11-03 22:55:30.496991635 +0000 UTC m=+1953.937528895"
Nov 03 22:55:30.603103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.603016    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"]
Nov 03 22:55:30.606459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.606433    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg"]
Nov 03 22:55:30.707623 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.707587    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" path="/var/lib/kubelet/pods/5a292b26-d09e-4f9b-8efd-53d8016b0475/volumes"
Nov 03 22:55:30.860346 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.860271    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:30.861729 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.861709    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 03 22:55:30.861783 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:30.861753    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 03 22:55:31.860160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:31.860128    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 03 22:55:31.860545 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:31.860198    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 03 22:55:32.513697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:32.506175    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:55:32.865750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:32.864996    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 03 22:55:32.865750 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:32.865062    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 03 22:55:33.864692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:33.861035    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 03 22:55:33.864692 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:33.861094    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 03 22:55:34.860388 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:34.860345    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 03 22:55:34.860942 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:34.860414    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 03 22:55:35.862710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:35.861913    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 03 22:55:35.862710 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:35.861985    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 03 22:55:37.113393 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:37.113353    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 03 22:55:37.114082 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:37.114047    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 03 22:55:37.672693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:37.671953    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:55:37.860448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:37.860413    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Nov 03 22:55:37.860760 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:37.860713    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Nov 03 22:55:39.418708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:39.418653    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:39.419337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:39.419305    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:39.420415 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:39.420308    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 22:55:46.045156 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:55:46.045109    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:55:46.051709 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:46.051682    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"]
Nov 03 22:55:47.030938 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.030904    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"]
Nov 03 22:55:47.031691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.031659    5647 cpu_manager.go:414] "containerMap: removing container" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy"
Nov 03 22:55:47.031743 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.031693    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy"
Nov 03 22:55:47.031842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.031833    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a292b26-d09e-4f9b-8efd-53d8016b0475" containerName="istio-proxy"
Nov 03 22:55:47.037015 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.036995    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.089142 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.089117    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-aacab9-openshift-default-dockercfg-p5sjp\""
Nov 03 22:55:47.128698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.128648    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"]
Nov 03 22:55:47.189837 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.189796    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istiod-ca-cert\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.190041 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.189978    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-envoy\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.190218 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.190189    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-data\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.190289 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.190238    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-credential-socket\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.190442 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.190419    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-socket\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.190505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.190458    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-podinfo\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.190505 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.190481    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-token\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.190618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.190548    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjh8n\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-kube-api-access-rjh8n\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.190618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.190599    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-certs\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.291922    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-socket\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.291956    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-podinfo\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.291975    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-token\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292025    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjh8n\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-kube-api-access-rjh8n\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292062    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-certs\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292093    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istiod-ca-cert\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292190    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-envoy\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292528 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292332    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-data\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292528 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292361    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-credential-socket\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292528 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292421    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-socket\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.292528 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292468    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-certs\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.293437 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.292814    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-credential-socket\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.293437 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.293332    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-data\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.293437 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.293339    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istiod-ca-cert\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.295158 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.295138    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-podinfo\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.295268 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.295249    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-envoy\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.352857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.352823    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-token\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.353591 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.353567    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjh8n\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-kube-api-access-rjh8n\") pod \"e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") " pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:47.651392 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:47.651060    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:48.233972 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:48.233940    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"]
Nov 03 22:55:48.234954 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:55:48.234929    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d0312c_9d69_435c_a0b1_05ef3b52e454.slice/crio-25306d868f644938e438788ed27ec3550ceb0e2eaf1e8dc6a783e1f1c57e670b WatchSource:0}: Error finding container 25306d868f644938e438788ed27ec3550ceb0e2eaf1e8dc6a783e1f1c57e670b: Status 404 returned error can't find the container with id 25306d868f644938e438788ed27ec3550ceb0e2eaf1e8dc6a783e1f1c57e670b
Nov 03 22:55:48.237786 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:48.237754    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:55:48.237874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:48.237822    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:55:48.237874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:48.237864    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:55:49.035296 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:49.035257    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" event={"ID":"a6d0312c-9d69-435c-a0b1-05ef3b52e454","Type":"ContainerStarted","Data":"243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a"}
Nov 03 22:55:49.035507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:49.035313    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" event={"ID":"a6d0312c-9d69-435c-a0b1-05ef3b52e454","Type":"ContainerStarted","Data":"25306d868f644938e438788ed27ec3550ceb0e2eaf1e8dc6a783e1f1c57e670b"}
Nov 03 22:55:49.204541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:49.204480    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" podStartSLOduration=3.204464613 podStartE2EDuration="3.204464613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:55:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:55:49.203342153 +0000 UTC m=+1972.643879425" watchObservedRunningTime="2026-11-03 22:55:49.204464613 +0000 UTC m=+1972.645001870"
Nov 03 22:55:49.656313 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:49.656268    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:49.660873 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:49.660846    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 03 22:55:49.661000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:49.660904    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 03 22:55:50.655889 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:50.655855    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 03 22:55:50.656068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:50.655907    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 03 22:55:51.052520 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.052487    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks" podUID="9f64e34b-2d3a-4298-b410-40d4d7a01334" containerName="payload-processing" containerID="cri-o://af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4" gracePeriod=25
Nov 03 22:55:51.416901 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.416557    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:55:51.557239 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.556846    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f64e34b-2d3a-4298-b410-40d4d7a01334-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f64e34b-2d3a-4298-b410-40d4d7a01334-plugins-config-volume\") pod \"9f64e34b-2d3a-4298-b410-40d4d7a01334\" (UID: \"9f64e34b-2d3a-4298-b410-40d4d7a01334\") "
Nov 03 22:55:51.557239 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.557043    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f64e34b-2d3a-4298-b410-40d4d7a01334-kube-api-access-zx9k5\" (UniqueName: \"kubernetes.io/projected/9f64e34b-2d3a-4298-b410-40d4d7a01334-kube-api-access-zx9k5\") pod \"9f64e34b-2d3a-4298-b410-40d4d7a01334\" (UID: \"9f64e34b-2d3a-4298-b410-40d4d7a01334\") "
Nov 03 22:55:51.557239 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.557185    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f64e34b-2d3a-4298-b410-40d4d7a01334-plugins-config-volume" pod "9f64e34b-2d3a-4298-b410-40d4d7a01334" (UID: "9f64e34b-2d3a-4298-b410-40d4d7a01334"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:55:51.557905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.557880    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f64e34b-2d3a-4298-b410-40d4d7a01334-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:51.559684 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.559649    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f64e34b-2d3a-4298-b410-40d4d7a01334-kube-api-access-zx9k5" pod "9f64e34b-2d3a-4298-b410-40d4d7a01334" (UID: "9f64e34b-2d3a-4298-b410-40d4d7a01334"). InnerVolumeSpecName "kube-api-access-zx9k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:55:51.656547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.656468    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 03 22:55:51.656806 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.656777    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 03 22:55:51.659595 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:51.659567    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx9k5\" (UniqueName: \"kubernetes.io/projected/9f64e34b-2d3a-4298-b410-40d4d7a01334-kube-api-access-zx9k5\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:55:52.058423 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.058383    5647 generic.go:356] "Generic (PLEG): container finished" podID="9f64e34b-2d3a-4298-b410-40d4d7a01334" containerID="af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4" exitCode=0
Nov 03 22:55:52.058847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.058427    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"
Nov 03 22:55:52.058847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.058476    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks" event={"ID":"9f64e34b-2d3a-4298-b410-40d4d7a01334","Type":"ContainerDied","Data":"af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4"}
Nov 03 22:55:52.058847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.058534    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks" event={"ID":"9f64e34b-2d3a-4298-b410-40d4d7a01334","Type":"ContainerDied","Data":"9b1ce8a1d1e65b3d6002549b01eba91c3033b5d24e9de615d6fa89e8f69d5f26"}
Nov 03 22:55:52.058847 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.058559    5647 scope.go:122] "RemoveContainer" containerID="af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4"
Nov 03 22:55:52.087930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.084877    5647 scope.go:122] "RemoveContainer" containerID="af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4"
Nov 03 22:55:52.087930 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:55:52.085289    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4\": container with ID starting with af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4 not found: ID does not exist" containerID="af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4"
Nov 03 22:55:52.087930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.085323    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4"} err="failed to get container status \"af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4\": rpc error: code = NotFound desc = could not find container \"af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4\": container with ID starting with af7eb1d5134c14f392eecf4fcca794fb218b8f518cb8df30c0791a5a4f729ab4 not found: ID does not exist"
Nov 03 22:55:52.226314 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.226287    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"]
Nov 03 22:55:52.299796 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.299761    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks"]
Nov 03 22:55:52.655973 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.655941    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 03 22:55:52.656193 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.656000    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 03 22:55:52.705804 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:52.705769    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f64e34b-2d3a-4298-b410-40d4d7a01334" path="/var/lib/kubelet/pods/9f64e34b-2d3a-4298-b410-40d4d7a01334/volumes"
Nov 03 22:55:53.655985 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:53.655953    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 03 22:55:53.656383 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:53.656003    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 03 22:55:54.661438 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:54.661410    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:54.661927 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:54.661905    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:54.662512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:54.662496    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:55:55.087906 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:55.087871    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"]
Nov 03 22:55:55.088123 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:55.088100    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy" containerID="cri-o://4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1" gracePeriod=30
Nov 03 22:55:55.774970 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:55.774933    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"]
Nov 03 22:55:55.775359 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:55.775130    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" containerID="cri-o://161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1" gracePeriod=30
Nov 03 22:55:57.558440 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:57.558398    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:55:57.558907 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:57.558466    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:55:58.519333 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:58.519301    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:55:58.519549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:55:58.519358    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:56:00.372524 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.372452    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:56:00.418056 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418027    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-socket\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418190 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418083    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-certs\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418190 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418139    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-kube-api-access-45k4t\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-kube-api-access-45k4t\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418190 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418173    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-credential-socket\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418351 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418263    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64740797-9960-4f13-9a00-a3c5e8eb11c7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64740797-9960-4f13-9a00-a3c5e8eb11c7-istiod-ca-cert\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418351 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418294    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-data\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418353    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-token\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-token\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418325    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-socket" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:00.418455 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418384    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-certs" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:00.418604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418475    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-podinfo\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418555    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-credential-socket" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:00.418604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418569    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-data" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:00.418604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418577    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-envoy\") pod \"64740797-9960-4f13-9a00-a3c5e8eb11c7\" (UID: \"64740797-9960-4f13-9a00-a3c5e8eb11c7\") "
Nov 03 22:56:00.418852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.418796    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64740797-9960-4f13-9a00-a3c5e8eb11c7-istiod-ca-cert" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:56:00.419568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.419542    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:00.419715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.419574    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:00.419715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.419588    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:00.419715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.419602    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64740797-9960-4f13-9a00-a3c5e8eb11c7-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:00.419715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.419617    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:00.420839 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.420807    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-kube-api-access-45k4t" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "kube-api-access-45k4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:56:00.420982 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.420961    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-envoy" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:00.421176 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.421151    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-podinfo" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:56:00.421273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.421187    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-token" pod "64740797-9960-4f13-9a00-a3c5e8eb11c7" (UID: "64740797-9960-4f13-9a00-a3c5e8eb11c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:56:00.521637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.521605    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45k4t\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-kube-api-access-45k4t\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:00.521637 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.521636    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:00.521937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.521648    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:00.521937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:00.521656    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64740797-9960-4f13-9a00-a3c5e8eb11c7-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.084554 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.084524    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:56:01.138113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138082    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-data\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138132    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istiod-ca-cert\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138255    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-certs\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138316 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138307    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-socket\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138354    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-kube-api-access-zrkrx\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-kube-api-access-zrkrx\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138395    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-data" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:01.138481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138415    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-envoy\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138445    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-podinfo\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138481 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138480    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-token\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-token\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138515    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-credential-socket\") pod \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\" (UID: \"358d8eaa-554b-4fd5-aaa1-39c81432b9c1\") "
Nov 03 22:56:01.138747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138566    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-certs" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:01.138747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138590    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-socket" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:01.138747 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138613    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istiod-ca-cert" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:56:01.138919 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.138809    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-credential-socket" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:01.139333 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.139300    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.139333 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.139329    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.139493 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.139344    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.139493 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.139356    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.139493 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.139367    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.140910 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.140889    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-podinfo" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:56:01.141029 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.140925    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-token" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:56:01.141296 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.141281    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-envoy" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:56:01.141353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.141305    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-kube-api-access-zrkrx" pod "358d8eaa-554b-4fd5-aaa1-39c81432b9c1" (UID: "358d8eaa-554b-4fd5-aaa1-39c81432b9c1"). InnerVolumeSpecName "kube-api-access-zrkrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:56:01.184135 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.184103    5647 generic.go:356] "Generic (PLEG): container finished" podID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerID="161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1" exitCode=0
Nov 03 22:56:01.184291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.184152    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"
Nov 03 22:56:01.184291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.184192    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" event={"ID":"358d8eaa-554b-4fd5-aaa1-39c81432b9c1","Type":"ContainerDied","Data":"161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1"}
Nov 03 22:56:01.184291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.184273    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5" event={"ID":"358d8eaa-554b-4fd5-aaa1-39c81432b9c1","Type":"ContainerDied","Data":"c768a1c8252c9e20efbc167a0e4b85a3e6ec33dfc122f8c014382e83eca5cc29"}
Nov 03 22:56:01.184420 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.184305    5647 scope.go:122] "RemoveContainer" containerID="161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1"
Nov 03 22:56:01.185915 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.185892    5647 generic.go:356] "Generic (PLEG): container finished" podID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerID="4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1" exitCode=0
Nov 03 22:56:01.186044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.185969    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"
Nov 03 22:56:01.186044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.186008    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" event={"ID":"64740797-9960-4f13-9a00-a3c5e8eb11c7","Type":"ContainerDied","Data":"4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1"}
Nov 03 22:56:01.186044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.186041    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x" event={"ID":"64740797-9960-4f13-9a00-a3c5e8eb11c7","Type":"ContainerDied","Data":"feacea3b4c379d768901afd2267a45f140ead2e6ad68b3c77c96c6a08eccbf78"}
Nov 03 22:56:01.195527 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.195510    5647 scope.go:122] "RemoveContainer" containerID="161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1"
Nov 03 22:56:01.195805 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:56:01.195785    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1\": container with ID starting with 161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1 not found: ID does not exist" containerID="161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1"
Nov 03 22:56:01.195876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.195813    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1"} err="failed to get container status \"161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1\": rpc error: code = NotFound desc = could not find container \"161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1\": container with ID starting with 161f1366eaea37876738864d879476dc12716dfccd8dadea4071bcb3f2eeffe1 not found: ID does not exist"
Nov 03 22:56:01.195876 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.195830    5647 scope.go:122] "RemoveContainer" containerID="4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1"
Nov 03 22:56:01.205563 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.205545    5647 scope.go:122] "RemoveContainer" containerID="4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1"
Nov 03 22:56:01.205960 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:56:01.205931    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1\": container with ID starting with 4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1 not found: ID does not exist" containerID="4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1"
Nov 03 22:56:01.206061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.205967    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1"} err="failed to get container status \"4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1\": rpc error: code = NotFound desc = could not find container \"4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1\": container with ID starting with 4bbcc45fd4507178cc4ab4c38a96567b026e6356d1574b4afc192080a85133e1 not found: ID does not exist"
Nov 03 22:56:01.241267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.241234    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrkrx\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-kube-api-access-zrkrx\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.241267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.241266    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.241556 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.241279    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.241556 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.241292    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/358d8eaa-554b-4fd5-aaa1-39c81432b9c1-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:01.319021 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.318952    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"]
Nov 03 22:56:01.334049 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.324445    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5"]
Nov 03 22:56:01.431454 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.431419    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"]
Nov 03 22:56:01.440190 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:01.440161    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x"]
Nov 03 22:56:02.697496 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:02.697455    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" path="/var/lib/kubelet/pods/358d8eaa-554b-4fd5-aaa1-39c81432b9c1/volumes"
Nov 03 22:56:02.698127 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:02.698109    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" path="/var/lib/kubelet/pods/64740797-9960-4f13-9a00-a3c5e8eb11c7/volumes"
Nov 03 22:56:16.081192 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:56:16.080444    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:56:39.958264 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:39.958220    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"]
Nov 03 22:56:39.959406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:39.959101    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s" podUID="23e0576c-6a95-4420-9f3a-13524619f118" containerName="maas-api" containerID="cri-o://823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334" gracePeriod=30
Nov 03 22:56:40.411717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.404207    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:56:40.493411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.491968    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23e0576c-6a95-4420-9f3a-13524619f118-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23e0576c-6a95-4420-9f3a-13524619f118-openshift-service-ca\") pod \"23e0576c-6a95-4420-9f3a-13524619f118\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") "
Nov 03 22:56:40.493411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.492083    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23e0576c-6a95-4420-9f3a-13524619f118-kube-api-access-l99ln\" (UniqueName: \"kubernetes.io/projected/23e0576c-6a95-4420-9f3a-13524619f118-kube-api-access-l99ln\") pod \"23e0576c-6a95-4420-9f3a-13524619f118\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") "
Nov 03 22:56:40.493411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.492153    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-metrics-certs\") pod \"23e0576c-6a95-4420-9f3a-13524619f118\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") "
Nov 03 22:56:40.493411 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.492240    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls\") pod \"23e0576c-6a95-4420-9f3a-13524619f118\" (UID: \"23e0576c-6a95-4420-9f3a-13524619f118\") "
Nov 03 22:56:40.495155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.494984    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23e0576c-6a95-4420-9f3a-13524619f118-openshift-service-ca" pod "23e0576c-6a95-4420-9f3a-13524619f118" (UID: "23e0576c-6a95-4420-9f3a-13524619f118"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:56:40.514257 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.514216    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-metrics-certs" pod "23e0576c-6a95-4420-9f3a-13524619f118" (UID: "23e0576c-6a95-4420-9f3a-13524619f118"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:56:40.514426 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.514371    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls" pod "23e0576c-6a95-4420-9f3a-13524619f118" (UID: "23e0576c-6a95-4420-9f3a-13524619f118"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:56:40.529792 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.529164    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e0576c-6a95-4420-9f3a-13524619f118-kube-api-access-l99ln" pod "23e0576c-6a95-4420-9f3a-13524619f118" (UID: "23e0576c-6a95-4420-9f3a-13524619f118"). InnerVolumeSpecName "kube-api-access-l99ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:56:40.561502 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.559364    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:56:40.594284 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.594199    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:40.594492 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.594477    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23e0576c-6a95-4420-9f3a-13524619f118-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:40.594585 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.594574    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23e0576c-6a95-4420-9f3a-13524619f118-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:40.594685 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.594655    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l99ln\" (UniqueName: \"kubernetes.io/projected/23e0576c-6a95-4420-9f3a-13524619f118-kube-api-access-l99ln\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:40.594880 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.594855    5647 generic.go:356] "Generic (PLEG): container finished" podID="23e0576c-6a95-4420-9f3a-13524619f118" containerID="823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334" exitCode=0
Nov 03 22:56:40.595135 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.595107    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s" event={"ID":"23e0576c-6a95-4420-9f3a-13524619f118","Type":"ContainerDied","Data":"823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334"}
Nov 03 22:56:40.595247 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.595165    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s" event={"ID":"23e0576c-6a95-4420-9f3a-13524619f118","Type":"ContainerDied","Data":"3e9cf2a492f3ec15df28b44add3c89fa6d660a52d3f0bf537a123888dc6b9340"}
Nov 03 22:56:40.595247 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.595157    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"
Nov 03 22:56:40.596076 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.595192    5647 scope.go:122] "RemoveContainer" containerID="823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334"
Nov 03 22:56:40.610824 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.610805    5647 scope.go:122] "RemoveContainer" containerID="823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334"
Nov 03 22:56:40.611151 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:56:40.611127    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334\": container with ID starting with 823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334 not found: ID does not exist" containerID="823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334"
Nov 03 22:56:40.611225 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.611163    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334"} err="failed to get container status \"823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334\": rpc error: code = NotFound desc = could not find container \"823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334\": container with ID starting with 823222cbb7257d8ff963a2dfca666626ba4990b172d9a81d5cd72d594e5a2334 not found: ID does not exist"
Nov 03 22:56:40.777300 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.777268    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"]
Nov 03 22:56:40.787242 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:40.787202    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a91d2d-866d6dfc7-6lb5s"]
Nov 03 22:56:42.698020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:42.697983    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23e0576c-6a95-4420-9f3a-13524619f118" path="/var/lib/kubelet/pods/23e0576c-6a95-4420-9f3a-13524619f118/volumes"
Nov 03 22:56:46.044770 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:56:46.044727    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:56:47.275380 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.273585    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"]
Nov 03 22:56:47.275380 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.273920    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj" podUID="b5629a4d-425c-4cc1-bbcc-6fed7714c29d" containerName="maas-api" containerID="cri-o://0e10750a6e426a46111328eec61332841e3342e89767da48af95c6e686002f4f" gracePeriod=30
Nov 03 22:56:47.810318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.809646    5647 generic.go:356] "Generic (PLEG): container finished" podID="b5629a4d-425c-4cc1-bbcc-6fed7714c29d" containerID="0e10750a6e426a46111328eec61332841e3342e89767da48af95c6e686002f4f" exitCode=0
Nov 03 22:56:47.810318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.809895    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj" event={"ID":"b5629a4d-425c-4cc1-bbcc-6fed7714c29d","Type":"ContainerDied","Data":"0e10750a6e426a46111328eec61332841e3342e89767da48af95c6e686002f4f"}
Nov 03 22:56:47.810318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.809951    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj" event={"ID":"b5629a4d-425c-4cc1-bbcc-6fed7714c29d","Type":"ContainerDied","Data":"2ca7e70bc8cf07aa00d0136078cc280356e05e6312d58d10413b421178a0b54d"}
Nov 03 22:56:47.810318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.809975    5647 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ca7e70bc8cf07aa00d0136078cc280356e05e6312d58d10413b421178a0b54d"
Nov 03 22:56:47.813069 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.813049    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:56:47.851209 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.851172    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls\") pod \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") "
Nov 03 22:56:47.851446 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.851278    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-metrics-certs\") pod \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") "
Nov 03 22:56:47.851446 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.851349    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-openshift-service-ca\") pod \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") "
Nov 03 22:56:47.851446 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.851437    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-kube-api-access-45dxc\" (UniqueName: \"kubernetes.io/projected/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-kube-api-access-45dxc\") pod \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\" (UID: \"b5629a4d-425c-4cc1-bbcc-6fed7714c29d\") "
Nov 03 22:56:47.855008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.854971    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-openshift-service-ca" pod "b5629a4d-425c-4cc1-bbcc-6fed7714c29d" (UID: "b5629a4d-425c-4cc1-bbcc-6fed7714c29d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:56:47.856569 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.856537    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-metrics-certs" pod "b5629a4d-425c-4cc1-bbcc-6fed7714c29d" (UID: "b5629a4d-425c-4cc1-bbcc-6fed7714c29d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:56:47.858101 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.858071    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls" pod "b5629a4d-425c-4cc1-bbcc-6fed7714c29d" (UID: "b5629a4d-425c-4cc1-bbcc-6fed7714c29d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:56:47.864130 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.864098    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-kube-api-access-45dxc" pod "b5629a4d-425c-4cc1-bbcc-6fed7714c29d" (UID: "b5629a4d-425c-4cc1-bbcc-6fed7714c29d"). InnerVolumeSpecName "kube-api-access-45dxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:56:47.956126 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.956087    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:47.956126 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.956126    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:47.956401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.956139    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:47.956401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:47.956153    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45dxc\" (UniqueName: \"kubernetes.io/projected/b5629a4d-425c-4cc1-bbcc-6fed7714c29d-kube-api-access-45dxc\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:56:48.817614 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:48.817579    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"
Nov 03 22:56:48.982942 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:48.982860    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"]
Nov 03 22:56:49.032168 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:49.032135    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-pmprj"]
Nov 03 22:56:50.697974 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:50.697944    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5629a4d-425c-4cc1-bbcc-6fed7714c29d" path="/var/lib/kubelet/pods/b5629a4d-425c-4cc1-bbcc-6fed7714c29d/volumes"
Nov 03 22:56:56.226696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:56.226645    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"]
Nov 03 22:56:56.227115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:56.226893    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy" containerID="cri-o://f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f" gracePeriod=30
Nov 03 22:56:59.384254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:56:59.384223    5647 scope.go:122] "RemoveContainer" containerID="0e10750a6e426a46111328eec61332841e3342e89767da48af95c6e686002f4f"
Nov 03 22:57:01.272788 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.272754    5647 patch_prober.go:29] interesting pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Nov 03 22:57:01.273162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.272820    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Nov 03 22:57:01.519581 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.519553    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:57:01.582294 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582260    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-istio-token\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-istio-token\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.582465 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582344    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/928babb1-e87e-4ce6-b257-d149c947c724-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928babb1-e87e-4ce6-b257-d149c947c724-istiod-ca-cert\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.582513 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582491    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-credential-socket\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.582548 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582531    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-certs\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.582591 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582560    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/928babb1-e87e-4ce6-b257-d149c947c724-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928babb1-e87e-4ce6-b257-d149c947c724-istio-podinfo\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.582717 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582662    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-data\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.582804 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582754    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-kube-api-access-x7bwl\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-kube-api-access-x7bwl\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.582804 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582768    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-credential-socket" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:01.582804 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582785    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-certs" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:01.582971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582818    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-socket\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.582971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582846    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/928babb1-e87e-4ce6-b257-d149c947c724-istiod-ca-cert" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:57:01.582971 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582893    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-envoy\") pod \"928babb1-e87e-4ce6-b257-d149c947c724\" (UID: \"928babb1-e87e-4ce6-b257-d149c947c724\") "
Nov 03 22:57:01.583116 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.582979    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-data" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:01.583321 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.583296    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-socket" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:01.583715 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.583691    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:01.583848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.583721    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:01.583848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.583735    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:01.583848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.583749    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:01.583848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.583762    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928babb1-e87e-4ce6-b257-d149c947c724-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:01.585065 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.585039    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-istio-token" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:01.585507 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.585481    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-kube-api-access-x7bwl" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "kube-api-access-x7bwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:01.585902 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.585881    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/928babb1-e87e-4ce6-b257-d149c947c724-istio-podinfo" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:57:01.586000 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.585876    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-envoy" pod "928babb1-e87e-4ce6-b257-d149c947c724" (UID: "928babb1-e87e-4ce6-b257-d149c947c724"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:01.685720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.685683    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:01.685720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.685714    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928babb1-e87e-4ce6-b257-d149c947c724-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:01.685720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.685725    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7bwl\" (UniqueName: \"kubernetes.io/projected/928babb1-e87e-4ce6-b257-d149c947c724-kube-api-access-x7bwl\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:01.685973 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:01.685740    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928babb1-e87e-4ce6-b257-d149c947c724-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:02.174718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.174651    5647 generic.go:356] "Generic (PLEG): container finished" podID="928babb1-e87e-4ce6-b257-d149c947c724" containerID="f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f" exitCode=0
Nov 03 22:57:02.174718 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.174703    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" event={"ID":"928babb1-e87e-4ce6-b257-d149c947c724","Type":"ContainerDied","Data":"f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f"}
Nov 03 22:57:02.174957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.174743    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"
Nov 03 22:57:02.174957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.174809    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df" event={"ID":"928babb1-e87e-4ce6-b257-d149c947c724","Type":"ContainerDied","Data":"170dd9c7069f52225a266dc4316e274d19fc577ed81e2ddaad97dc8842d00dac"}
Nov 03 22:57:02.174957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.174824    5647 scope.go:122] "RemoveContainer" containerID="f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f"
Nov 03 22:57:02.187402 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.187381    5647 scope.go:122] "RemoveContainer" containerID="e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39"
Nov 03 22:57:02.197479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.197458    5647 scope.go:122] "RemoveContainer" containerID="f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f"
Nov 03 22:57:02.197811 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:57:02.197785    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f\": container with ID starting with f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f not found: ID does not exist" containerID="f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f"
Nov 03 22:57:02.197869 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.197823    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f"} err="failed to get container status \"f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f\": rpc error: code = NotFound desc = could not find container \"f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f\": container with ID starting with f8afe30d9cc34b2b14d11710d23b1516f7148a6ed6c43e4f819c42894a1d214f not found: ID does not exist"
Nov 03 22:57:02.197869 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.197841    5647 scope.go:122] "RemoveContainer" containerID="e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39"
Nov 03 22:57:02.198146 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:57:02.198128    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39\": container with ID starting with e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39 not found: ID does not exist" containerID="e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39"
Nov 03 22:57:02.198212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.198150    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39"} err="failed to get container status \"e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39\": rpc error: code = NotFound desc = could not find container \"e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39\": container with ID starting with e1ad5e86942903966abbcf13df52f0f05d73518c40058c666f5f269ec5321e39 not found: ID does not exist"
Nov 03 22:57:02.339836 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.339796    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"]
Nov 03 22:57:02.389276 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.389238    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df"]
Nov 03 22:57:02.698168 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:02.698127    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="928babb1-e87e-4ce6-b257-d149c947c724" path="/var/lib/kubelet/pods/928babb1-e87e-4ce6-b257-d149c947c724/volumes"
Nov 03 22:57:07.209139 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.209108    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"]
Nov 03 22:57:07.211197 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.211173    5647 cpu_manager.go:414] "containerMap: removing container" podUID="9f64e34b-2d3a-4298-b410-40d4d7a01334" containerName="payload-processing"
Nov 03 22:57:07.211369 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.211355    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f64e34b-2d3a-4298-b410-40d4d7a01334" containerName="payload-processing"
Nov 03 22:57:07.211521 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.211510    5647 cpu_manager.go:414] "containerMap: removing container" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy"
Nov 03 22:57:07.211615 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.211605    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy"
Nov 03 22:57:07.211726 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.211716    5647 cpu_manager.go:414] "containerMap: removing container" podUID="b5629a4d-425c-4cc1-bbcc-6fed7714c29d" containerName="maas-api"
Nov 03 22:57:07.211832 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.211821    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5629a4d-425c-4cc1-bbcc-6fed7714c29d" containerName="maas-api"
Nov 03 22:57:07.211947 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.211937    5647 cpu_manager.go:414] "containerMap: removing container" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy"
Nov 03 22:57:07.212038 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.212028    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy"
Nov 03 22:57:07.212147 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.212138    5647 cpu_manager.go:414] "containerMap: removing container" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy"
Nov 03 22:57:07.212233 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.212224    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy"
Nov 03 22:57:07.212327 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.212318    5647 cpu_manager.go:414] "containerMap: removing container" podUID="23e0576c-6a95-4420-9f3a-13524619f118" containerName="maas-api"
Nov 03 22:57:07.212420 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.212411    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23e0576c-6a95-4420-9f3a-13524619f118" containerName="maas-api"
Nov 03 22:57:07.212527 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.212518    5647 cpu_manager.go:414] "containerMap: removing container" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy"
Nov 03 22:57:07.212616 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.212606    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy"
Nov 03 22:57:07.213043 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.213016    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="358d8eaa-554b-4fd5-aaa1-39c81432b9c1" containerName="istio-proxy"
Nov 03 22:57:07.213175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.213165    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5629a4d-425c-4cc1-bbcc-6fed7714c29d" containerName="maas-api"
Nov 03 22:57:07.213704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.213685    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy"
Nov 03 22:57:07.213848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.213837    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="928babb1-e87e-4ce6-b257-d149c947c724" containerName="istio-proxy"
Nov 03 22:57:07.213930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.213921    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="23e0576c-6a95-4420-9f3a-13524619f118" containerName="maas-api"
Nov 03 22:57:07.214011 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.214002    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="64740797-9960-4f13-9a00-a3c5e8eb11c7" containerName="istio-proxy"
Nov 03 22:57:07.214115 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.214105    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f64e34b-2d3a-4298-b410-40d4d7a01334" containerName="payload-processing"
Nov 03 22:57:07.220769 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.220717    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:07.232173 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.232142    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"]
Nov 03 22:57:07.276544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.276509    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-efd39a-dockercfg-7dzkc\""
Nov 03 22:57:07.276544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.276528    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-efd39a\""
Nov 03 22:57:07.317649 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.317616    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbz7x\" (UniqueName: \"kubernetes.io/projected/f62143b4-dae9-42b0-a6a1-e042d8e71257-kube-api-access-bbz7x\") pod \"payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp\" (UID: \"f62143b4-dae9-42b0-a6a1-e042d8e71257\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:07.317870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.317791    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f62143b4-dae9-42b0-a6a1-e042d8e71257-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp\" (UID: \"f62143b4-dae9-42b0-a6a1-e042d8e71257\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:07.321806 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.321789    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"]
Nov 03 22:57:07.326282 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.326262    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:07.372713 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.372657    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"]
Nov 03 22:57:07.426694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.422981    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbz7x\" (UniqueName: \"kubernetes.io/projected/f62143b4-dae9-42b0-a6a1-e042d8e71257-kube-api-access-bbz7x\") pod \"payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp\" (UID: \"f62143b4-dae9-42b0-a6a1-e042d8e71257\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:07.426694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.423147    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f62143b4-dae9-42b0-a6a1-e042d8e71257-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp\" (UID: \"f62143b4-dae9-42b0-a6a1-e042d8e71257\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:07.426694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.424471    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f62143b4-dae9-42b0-a6a1-e042d8e71257-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp\" (UID: \"f62143b4-dae9-42b0-a6a1-e042d8e71257\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:07.503742 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.502688    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbz7x\" (UniqueName: \"kubernetes.io/projected/f62143b4-dae9-42b0-a6a1-e042d8e71257-kube-api-access-bbz7x\") pod \"payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp\" (UID: \"f62143b4-dae9-42b0-a6a1-e042d8e71257\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:07.527693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.525186    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxftw\" (UniqueName: \"kubernetes.io/projected/67837717-e0d8-445d-82ae-7392f7a104c3-kube-api-access-xxftw\") pod \"payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg\" (UID: \"67837717-e0d8-445d-82ae-7392f7a104c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:07.527693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.525402    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67837717-e0d8-445d-82ae-7392f7a104c3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg\" (UID: \"67837717-e0d8-445d-82ae-7392f7a104c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:07.552701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.545971    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:07.630972 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.630929    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxftw\" (UniqueName: \"kubernetes.io/projected/67837717-e0d8-445d-82ae-7392f7a104c3-kube-api-access-xxftw\") pod \"payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg\" (UID: \"67837717-e0d8-445d-82ae-7392f7a104c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:07.631155 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.631069    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67837717-e0d8-445d-82ae-7392f7a104c3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg\" (UID: \"67837717-e0d8-445d-82ae-7392f7a104c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:07.631953 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.631925    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67837717-e0d8-445d-82ae-7392f7a104c3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg\" (UID: \"67837717-e0d8-445d-82ae-7392f7a104c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:07.695160 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.693929    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxftw\" (UniqueName: \"kubernetes.io/projected/67837717-e0d8-445d-82ae-7392f7a104c3-kube-api-access-xxftw\") pod \"payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg\" (UID: \"67837717-e0d8-445d-82ae-7392f7a104c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:07.942141 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:07.942049    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:08.036643 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:57:08.024219    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62143b4_dae9_42b0_a6a1_e042d8e71257.slice/crio-8262744cc470c088ac2cb39e3068e44a93a5a226fcc013511ee8f348ade23264 WatchSource:0}: Error finding container 8262744cc470c088ac2cb39e3068e44a93a5a226fcc013511ee8f348ade23264: Status 404 returned error can't find the container with id 8262744cc470c088ac2cb39e3068e44a93a5a226fcc013511ee8f348ade23264
Nov 03 22:57:08.039259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:08.039226    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"]
Nov 03 22:57:08.289998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:08.289963    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp" event={"ID":"f62143b4-dae9-42b0-a6a1-e042d8e71257","Type":"ContainerStarted","Data":"8262744cc470c088ac2cb39e3068e44a93a5a226fcc013511ee8f348ade23264"}
Nov 03 22:57:08.336638 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:57:08.336606    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67837717_e0d8_445d_82ae_7392f7a104c3.slice/crio-9e87b2c0bf7622a43c62f4861e075a6f1a18945cdda67d7678c7558cd639aff0 WatchSource:0}: Error finding container 9e87b2c0bf7622a43c62f4861e075a6f1a18945cdda67d7678c7558cd639aff0: Status 404 returned error can't find the container with id 9e87b2c0bf7622a43c62f4861e075a6f1a18945cdda67d7678c7558cd639aff0
Nov 03 22:57:08.336821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:08.336762    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"]
Nov 03 22:57:09.378929 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:09.378854    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg" event={"ID":"67837717-e0d8-445d-82ae-7392f7a104c3","Type":"ContainerStarted","Data":"0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e"}
Nov 03 22:57:09.379605 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:09.379555    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg" event={"ID":"67837717-e0d8-445d-82ae-7392f7a104c3","Type":"ContainerStarted","Data":"9e87b2c0bf7622a43c62f4861e075a6f1a18945cdda67d7678c7558cd639aff0"}
Nov 03 22:57:09.381098 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:09.381045    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:09.383822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:09.383800    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp" event={"ID":"f62143b4-dae9-42b0-a6a1-e042d8e71257","Type":"ContainerStarted","Data":"3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81"}
Nov 03 22:57:09.384821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:09.384802    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:09.763708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:09.759855    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp" podStartSLOduration=2.759834794 podStartE2EDuration="2.759834794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:57:09.747535012 +0000 UTC m=+2053.188072272" watchObservedRunningTime="2026-11-03 22:57:09.759834794 +0000 UTC m=+2053.200372053"
Nov 03 22:57:09.763708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:09.760280    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg" podStartSLOduration=2.76026903 podStartE2EDuration="2.76026903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:57:09.522355385 +0000 UTC m=+2052.962892645" watchObservedRunningTime="2026-11-03 22:57:09.76026903 +0000 UTC m=+2053.200806290"
Nov 03 22:57:10.950696 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:10.942766    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 22:57:16.044253 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:57:16.044212    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:57:20.301809 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.301770    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"]
Nov 03 22:57:20.302297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.301974    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc" podUID="f2a20bfd-e59f-44ad-9463-6d3d54ef713b" containerName="maas-api" containerID="cri-o://55f8abcb7853da15299e6ad2f46d9fb36e1a4bc93872ced269b3865f53abcc4e" gracePeriod=30
Nov 03 22:57:20.646902 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.646871    5647 generic.go:356] "Generic (PLEG): container finished" podID="f2a20bfd-e59f-44ad-9463-6d3d54ef713b" containerID="55f8abcb7853da15299e6ad2f46d9fb36e1a4bc93872ced269b3865f53abcc4e" exitCode=0
Nov 03 22:57:20.647077 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.646950    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc" event={"ID":"f2a20bfd-e59f-44ad-9463-6d3d54ef713b","Type":"ContainerDied","Data":"55f8abcb7853da15299e6ad2f46d9fb36e1a4bc93872ced269b3865f53abcc4e"}
Nov 03 22:57:20.679330 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.679275    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:57:20.806542 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.806499    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-openshift-service-ca\") pod \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") "
Nov 03 22:57:20.807128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.807032    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-kube-api-access-42brw\" (UniqueName: \"kubernetes.io/projected/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-kube-api-access-42brw\") pod \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") "
Nov 03 22:57:20.807376 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.807358    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-metrics-certs\") pod \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") "
Nov 03 22:57:20.807614 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.807596    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls\") pod \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\" (UID: \"f2a20bfd-e59f-44ad-9463-6d3d54ef713b\") "
Nov 03 22:57:20.810360 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.810319    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-openshift-service-ca" pod "f2a20bfd-e59f-44ad-9463-6d3d54ef713b" (UID: "f2a20bfd-e59f-44ad-9463-6d3d54ef713b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:57:20.850977 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.850853    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls" pod "f2a20bfd-e59f-44ad-9463-6d3d54ef713b" (UID: "f2a20bfd-e59f-44ad-9463-6d3d54ef713b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:57:20.855708 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.851446    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-metrics-certs" pod "f2a20bfd-e59f-44ad-9463-6d3d54ef713b" (UID: "f2a20bfd-e59f-44ad-9463-6d3d54ef713b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:57:20.867070 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.866762    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-kube-api-access-42brw" pod "f2a20bfd-e59f-44ad-9463-6d3d54ef713b" (UID: "f2a20bfd-e59f-44ad-9463-6d3d54ef713b"). InnerVolumeSpecName "kube-api-access-42brw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:20.910330 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.910246    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:20.910330 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.910283    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42brw\" (UniqueName: \"kubernetes.io/projected/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-kube-api-access-42brw\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:20.910330 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.910297    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:20.910330 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:20.910310    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a20bfd-e59f-44ad-9463-6d3d54ef713b-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:21.041459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.041425    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"]
Nov 03 22:57:21.042259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.042243    5647 cpu_manager.go:414] "containerMap: removing container" podUID="f2a20bfd-e59f-44ad-9463-6d3d54ef713b" containerName="maas-api"
Nov 03 22:57:21.042302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.042262    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2a20bfd-e59f-44ad-9463-6d3d54ef713b" containerName="maas-api"
Nov 03 22:57:21.042395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.042386    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2a20bfd-e59f-44ad-9463-6d3d54ef713b" containerName="maas-api"
Nov 03 22:57:21.046958 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.046931    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.268694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.250212    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f8cd0c-openshift-default-dockercfg-22lxw\""
Nov 03 22:57:21.297694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.296942    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"]
Nov 03 22:57:21.398693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398346    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cjbc\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-kube-api-access-4cjbc\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.398693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398471    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdd92147-66a8-49de-9612-468a9c71cac1-istio-podinfo\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.398693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398589    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-certs\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.398693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398651    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdd92147-66a8-49de-9612-468a9c71cac1-istiod-ca-cert\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.398693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398705    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-socket\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.399589 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398780    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-envoy\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.399589 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398807    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-data\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.399589 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398830    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-istio-token\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.399589 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.398862    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-credential-socket\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501138    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-certs\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501224    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdd92147-66a8-49de-9612-468a9c71cac1-istiod-ca-cert\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501268    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-socket\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501343    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-envoy\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501371    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-data\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501395    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-istio-token\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501423    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-credential-socket\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501570    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cjbc\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-kube-api-access-4cjbc\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.501693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.501697    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdd92147-66a8-49de-9612-468a9c71cac1-istio-podinfo\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.529512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.527491    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-data\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.529512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.527800    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-socket\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.529512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.527855    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdd92147-66a8-49de-9612-468a9c71cac1-istiod-ca-cert\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.529512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.528062    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-credential-socket\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.529512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.528144    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-certs\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.533207 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.533178    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdd92147-66a8-49de-9612-468a9c71cac1-istio-podinfo\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.533598 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.533573    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-envoy\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.585433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.585401    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 22:57:21.597008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.596964    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-istio-token\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.597278 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.597249    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 22:57:21.600468 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.600442    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cjbc\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-kube-api-access-4cjbc\") pod \"e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") " pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.678306 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.675390    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:21.701284 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.701254    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"
Nov 03 22:57:21.701460 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.701259    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc" event={"ID":"f2a20bfd-e59f-44ad-9463-6d3d54ef713b","Type":"ContainerDied","Data":"99403e6dee1387fe8d28283c157705012bf4d3145dee94fa165a58af64f99e52"}
Nov 03 22:57:21.701460 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.701400    5647 scope.go:122] "RemoveContainer" containerID="55f8abcb7853da15299e6ad2f46d9fb36e1a4bc93872ced269b3865f53abcc4e"
Nov 03 22:57:21.991997 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:21.991932    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"]
Nov 03 22:57:22.000881 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.000847    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ea1be0-56876ddcb4-2q6vc"]
Nov 03 22:57:22.105810 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.105779    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"]
Nov 03 22:57:22.107202 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:57:22.107180    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdd92147_66a8_49de_9612_468a9c71cac1.slice/crio-d3048b3dfe441e8db8fba60731a81c8533b95d12527b9c46194c094365559a05 WatchSource:0}: Error finding container d3048b3dfe441e8db8fba60731a81c8533b95d12527b9c46194c094365559a05: Status 404 returned error can't find the container with id d3048b3dfe441e8db8fba60731a81c8533b95d12527b9c46194c094365559a05
Nov 03 22:57:22.110423 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.110373    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:57:22.110523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.110478    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:57:22.110523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.110515    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:57:22.700987 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.700946    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2a20bfd-e59f-44ad-9463-6d3d54ef713b" path="/var/lib/kubelet/pods/f2a20bfd-e59f-44ad-9463-6d3d54ef713b/volumes"
Nov 03 22:57:22.716295 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.716256    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" event={"ID":"bdd92147-66a8-49de-9612-468a9c71cac1","Type":"ContainerStarted","Data":"497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e"}
Nov 03 22:57:22.716497 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.716319    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" event={"ID":"bdd92147-66a8-49de-9612-468a9c71cac1","Type":"ContainerStarted","Data":"d3048b3dfe441e8db8fba60731a81c8533b95d12527b9c46194c094365559a05"}
Nov 03 22:57:22.853176 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:22.852982    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podStartSLOduration=2.85296237 podStartE2EDuration="2.85296237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:57:22.848413444 +0000 UTC m=+2066.288950725" watchObservedRunningTime="2026-11-03 22:57:22.85296237 +0000 UTC m=+2066.293499629"
Nov 03 22:57:23.676518 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:23.676482    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:23.682059 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:23.682032    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:23.725464 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:23.725435    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:23.726519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:23.726501    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 22:57:29.603324 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.603287    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"]
Nov 03 22:57:29.608268 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.608244    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:29.625038 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.625011    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfb7q\" (UniqueName: \"kubernetes.io/projected/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-kube-api-access-lfb7q\") pod \"payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55\" (UID: \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:29.625227 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.625202    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55\" (UID: \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:29.710638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.710604    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-f8cd0c-dockercfg-kc29r\""
Nov 03 22:57:29.710912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.710656    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-f8cd0c\""
Nov 03 22:57:29.716905 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.716871    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"]
Nov 03 22:57:29.726844 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.726791    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55\" (UID: \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:29.727028 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.726985    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfb7q\" (UniqueName: \"kubernetes.io/projected/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-kube-api-access-lfb7q\") pod \"payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55\" (UID: \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:29.728049 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.728023    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55\" (UID: \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:29.790874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.790839    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfb7q\" (UniqueName: \"kubernetes.io/projected/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-kube-api-access-lfb7q\") pod \"payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55\" (UID: \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:29.928824 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:29.924988    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:30.502815 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:30.502788    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"]
Nov 03 22:57:30.504919 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:57:30.504893    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da35a7f_116b_4518_b27b_4fc35a4c7f9e.slice/crio-dd91049adeaf14097653ef76ca53fd1da0e2efd07d9412850983d5d41cb4450a WatchSource:0}: Error finding container dd91049adeaf14097653ef76ca53fd1da0e2efd07d9412850983d5d41cb4450a: Status 404 returned error can't find the container with id dd91049adeaf14097653ef76ca53fd1da0e2efd07d9412850983d5d41cb4450a
Nov 03 22:57:30.786961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:30.786867    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55" event={"ID":"5da35a7f-116b-4518-b27b-4fc35a4c7f9e","Type":"ContainerStarted","Data":"9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee"}
Nov 03 22:57:30.786961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:30.786918    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55" event={"ID":"5da35a7f-116b-4518-b27b-4fc35a4c7f9e","Type":"ContainerStarted","Data":"dd91049adeaf14097653ef76ca53fd1da0e2efd07d9412850983d5d41cb4450a"}
Nov 03 22:57:30.787764 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:30.787077    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:30.904056 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:30.904001    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55" podStartSLOduration=1.9039877779999999 podStartE2EDuration="1.903987778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:57:30.902623038 +0000 UTC m=+2074.343160300" watchObservedRunningTime="2026-11-03 22:57:30.903987778 +0000 UTC m=+2074.344525038"
Nov 03 22:57:41.799354 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:41.799320    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 22:57:42.190553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:42.190475    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"]
Nov 03 22:57:45.953348 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:45.953173    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"]
Nov 03 22:57:45.953928 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:45.953421    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy" containerID="cri-o://8e81906d0a8740af759617d696612b163effd5625249146918f429599adc9afb" gracePeriod=30
Nov 03 22:57:46.254701 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:57:46.252898    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:57:47.191022 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:47.190954    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr" podUID="b63a92a7-bd36-435b-b0af-1967f048af8b" containerName="payload-processing" containerID="cri-o://43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477" gracePeriod=25
Nov 03 22:57:47.471223 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:47.471197    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:57:47.528289 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:47.528257    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b63a92a7-bd36-435b-b0af-1967f048af8b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b63a92a7-bd36-435b-b0af-1967f048af8b-plugins-config-volume\") pod \"b63a92a7-bd36-435b-b0af-1967f048af8b\" (UID: \"b63a92a7-bd36-435b-b0af-1967f048af8b\") "
Nov 03 22:57:47.528488 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:47.528303    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b63a92a7-bd36-435b-b0af-1967f048af8b-kube-api-access-ks2t2\" (UniqueName: \"kubernetes.io/projected/b63a92a7-bd36-435b-b0af-1967f048af8b-kube-api-access-ks2t2\") pod \"b63a92a7-bd36-435b-b0af-1967f048af8b\" (UID: \"b63a92a7-bd36-435b-b0af-1967f048af8b\") "
Nov 03 22:57:47.528648 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:47.528624    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b63a92a7-bd36-435b-b0af-1967f048af8b-plugins-config-volume" pod "b63a92a7-bd36-435b-b0af-1967f048af8b" (UID: "b63a92a7-bd36-435b-b0af-1967f048af8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:57:47.528879 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:47.528859    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b63a92a7-bd36-435b-b0af-1967f048af8b-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:47.530660 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:47.530638    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b63a92a7-bd36-435b-b0af-1967f048af8b-kube-api-access-ks2t2" pod "b63a92a7-bd36-435b-b0af-1967f048af8b" (UID: "b63a92a7-bd36-435b-b0af-1967f048af8b"). InnerVolumeSpecName "kube-api-access-ks2t2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:47.630549 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:47.630513    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks2t2\" (UniqueName: \"kubernetes.io/projected/b63a92a7-bd36-435b-b0af-1967f048af8b-kube-api-access-ks2t2\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:48.130837 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.130800    5647 generic.go:356] "Generic (PLEG): container finished" podID="b63a92a7-bd36-435b-b0af-1967f048af8b" containerID="43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477" exitCode=0
Nov 03 22:57:48.131016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.130846    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"
Nov 03 22:57:48.131016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.130884    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr" event={"ID":"b63a92a7-bd36-435b-b0af-1967f048af8b","Type":"ContainerDied","Data":"43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477"}
Nov 03 22:57:48.131016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.130946    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr" event={"ID":"b63a92a7-bd36-435b-b0af-1967f048af8b","Type":"ContainerDied","Data":"86c98660032065117521af0367c2dc029e55de82623328cad78b0d220bbb8d46"}
Nov 03 22:57:48.131016 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.130971    5647 scope.go:122] "RemoveContainer" containerID="43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477"
Nov 03 22:57:48.142191 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.142167    5647 scope.go:122] "RemoveContainer" containerID="43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477"
Nov 03 22:57:48.142519 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:57:48.142496    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477\": container with ID starting with 43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477 not found: ID does not exist" containerID="43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477"
Nov 03 22:57:48.142588 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.142532    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477"} err="failed to get container status \"43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477\": rpc error: code = NotFound desc = could not find container \"43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477\": container with ID starting with 43a41e380f3582b6defb1a84bf035b02d2bc3c0012685c1efcfbee8b9034f477 not found: ID does not exist"
Nov 03 22:57:48.296064 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.296031    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"]
Nov 03 22:57:48.344870 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.344829    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr"]
Nov 03 22:57:48.698865 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:48.698828    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b63a92a7-bd36-435b-b0af-1967f048af8b" path="/var/lib/kubelet/pods/b63a92a7-bd36-435b-b0af-1967f048af8b/volumes"
Nov 03 22:57:51.160433 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.160403    5647 generic.go:356] "Generic (PLEG): container finished" podID="63034fce-49e9-40a2-ae8b-818725c20288" containerID="8e81906d0a8740af759617d696612b163effd5625249146918f429599adc9afb" exitCode=0
Nov 03 22:57:51.160917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.160606    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" event={"ID":"63034fce-49e9-40a2-ae8b-818725c20288","Type":"ContainerDied","Data":"8e81906d0a8740af759617d696612b163effd5625249146918f429599adc9afb"}
Nov 03 22:57:51.160917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.160711    5647 scope.go:122] "RemoveContainer" containerID="78d24639b4b8ad9fb47fdadb4300e81f61883b10b802a195d58a4e62e65edf01"
Nov 03 22:57:51.247511 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.247479    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:57:51.294812 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.294778    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-certs\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.294998 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.294928    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-istio-token\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-istio-token\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.295066 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295031    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63034fce-49e9-40a2-ae8b-818725c20288-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63034fce-49e9-40a2-ae8b-818725c20288-istiod-ca-cert\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.295131 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295101    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-certs" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:51.295131 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295120    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-data\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.295237 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295201    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-kube-api-access-rlxch\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-kube-api-access-rlxch\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.295292 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295242    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/63034fce-49e9-40a2-ae8b-818725c20288-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63034fce-49e9-40a2-ae8b-818725c20288-istio-podinfo\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.295349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295292    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-credential-socket\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.295349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295325    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-data" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:51.295349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295332    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-envoy\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.295511 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295428    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-socket\") pod \"63034fce-49e9-40a2-ae8b-818725c20288\" (UID: \"63034fce-49e9-40a2-ae8b-818725c20288\") "
Nov 03 22:57:51.295632 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295423    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63034fce-49e9-40a2-ae8b-818725c20288-istiod-ca-cert" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:57:51.295632 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295534    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-credential-socket" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:51.295850 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.295817    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-socket" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:51.296178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.296148    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:51.296178 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.296172    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:51.296332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.296186    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:51.296332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.296198    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63034fce-49e9-40a2-ae8b-818725c20288-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:51.296332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.296211    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:51.297911 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.297884    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-envoy" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:57:51.297911 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.297902    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-kube-api-access-rlxch" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "kube-api-access-rlxch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:51.298061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.297925    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-istio-token" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:57:51.298249 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.298231    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63034fce-49e9-40a2-ae8b-818725c20288-istio-podinfo" pod "63034fce-49e9-40a2-ae8b-818725c20288" (UID: "63034fce-49e9-40a2-ae8b-818725c20288"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:57:51.397957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.397926    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlxch\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-kube-api-access-rlxch\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:51.397957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.397953    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63034fce-49e9-40a2-ae8b-818725c20288-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:51.397957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.397963    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63034fce-49e9-40a2-ae8b-818725c20288-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:51.398195 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:51.397971    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63034fce-49e9-40a2-ae8b-818725c20288-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:57:52.169344 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:52.169305    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z" event={"ID":"63034fce-49e9-40a2-ae8b-818725c20288","Type":"ContainerDied","Data":"ef9d0bc3901673596ee5548edffc6344ecfc9bead012fba55e911061cf0922ec"}
Nov 03 22:57:52.169802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:52.169403    5647 scope.go:122] "RemoveContainer" containerID="8e81906d0a8740af759617d696612b163effd5625249146918f429599adc9afb"
Nov 03 22:57:52.169802 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:52.169319    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"
Nov 03 22:57:52.334592 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:52.334562    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"]
Nov 03 22:57:52.337836 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:52.337810    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z"]
Nov 03 22:57:52.697291 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:52.697254    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63034fce-49e9-40a2-ae8b-818725c20288" path="/var/lib/kubelet/pods/63034fce-49e9-40a2-ae8b-818725c20288/volumes"
Nov 03 22:57:55.171216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:55.171180    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:57:57.020361 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:57.020331    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:57:57.024852 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:57.024829    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 22:57:59.331500 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:59.331464    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"]
Nov 03 22:57:59.450157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:57:59.443190    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"]
Nov 03 22:58:04.332602 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.332561    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq" podUID="9f615715-b0f7-4754-b079-f998dd0cf49d" containerName="payload-processing" containerID="cri-o://5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f" gracePeriod=25
Nov 03 22:58:04.443562 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.443518    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn" podUID="c1399892-8ce0-410d-9bd8-d1a2a40df2e8" containerName="payload-pre-processing" containerID="cri-o://a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846" gracePeriod=25
Nov 03 22:58:04.640111 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.639816    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:58:04.734767 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.734736    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f615715-b0f7-4754-b079-f998dd0cf49d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f615715-b0f7-4754-b079-f998dd0cf49d-plugins-config-volume\") pod \"9f615715-b0f7-4754-b079-f998dd0cf49d\" (UID: \"9f615715-b0f7-4754-b079-f998dd0cf49d\") "
Nov 03 22:58:04.735074 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.735054    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f615715-b0f7-4754-b079-f998dd0cf49d-kube-api-access-twqhd\" (UniqueName: \"kubernetes.io/projected/9f615715-b0f7-4754-b079-f998dd0cf49d-kube-api-access-twqhd\") pod \"9f615715-b0f7-4754-b079-f998dd0cf49d\" (UID: \"9f615715-b0f7-4754-b079-f998dd0cf49d\") "
Nov 03 22:58:04.735231 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.735121    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f615715-b0f7-4754-b079-f998dd0cf49d-plugins-config-volume" pod "9f615715-b0f7-4754-b079-f998dd0cf49d" (UID: "9f615715-b0f7-4754-b079-f998dd0cf49d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:04.735760 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.735737    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f615715-b0f7-4754-b079-f998dd0cf49d-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:04.737761 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.737741    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f615715-b0f7-4754-b079-f998dd0cf49d-kube-api-access-twqhd" pod "9f615715-b0f7-4754-b079-f998dd0cf49d" (UID: "9f615715-b0f7-4754-b079-f998dd0cf49d"). InnerVolumeSpecName "kube-api-access-twqhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:04.748202 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.748178    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:58:04.823211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.823169    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"]
Nov 03 22:58:04.824948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.824801    5647 cpu_manager.go:414] "containerMap: removing container" podUID="b63a92a7-bd36-435b-b0af-1967f048af8b" containerName="payload-processing"
Nov 03 22:58:04.824948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.824826    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b63a92a7-bd36-435b-b0af-1967f048af8b" containerName="payload-processing"
Nov 03 22:58:04.824948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.824877    5647 cpu_manager.go:414] "containerMap: removing container" podUID="9f615715-b0f7-4754-b079-f998dd0cf49d" containerName="payload-processing"
Nov 03 22:58:04.824948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.824886    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f615715-b0f7-4754-b079-f998dd0cf49d" containerName="payload-processing"
Nov 03 22:58:04.824948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.824896    5647 cpu_manager.go:414] "containerMap: removing container" podUID="c1399892-8ce0-410d-9bd8-d1a2a40df2e8" containerName="payload-pre-processing"
Nov 03 22:58:04.824948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.824905    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1399892-8ce0-410d-9bd8-d1a2a40df2e8" containerName="payload-pre-processing"
Nov 03 22:58:04.824948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.824934    5647 cpu_manager.go:414] "containerMap: removing container" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy"
Nov 03 22:58:04.824948 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.824941    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy"
Nov 03 22:58:04.825587 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.825012    5647 cpu_manager.go:414] "containerMap: removing container" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy"
Nov 03 22:58:04.825587 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.825020    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy"
Nov 03 22:58:04.825587 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.825305    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="b63a92a7-bd36-435b-b0af-1967f048af8b" containerName="payload-processing"
Nov 03 22:58:04.825587 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.825336    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f615715-b0f7-4754-b079-f998dd0cf49d" containerName="payload-processing"
Nov 03 22:58:04.825587 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.825353    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1399892-8ce0-410d-9bd8-d1a2a40df2e8" containerName="payload-pre-processing"
Nov 03 22:58:04.825587 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.825369    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy"
Nov 03 22:58:04.825587 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.825424    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="63034fce-49e9-40a2-ae8b-818725c20288" containerName="istio-proxy"
Nov 03 22:58:04.830520 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.830496    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:04.837032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.837006    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-plugins-config-volume\") pod \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\" (UID: \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\") "
Nov 03 22:58:04.837149 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.837064    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-kube-api-access-g6fdj\" (UniqueName: \"kubernetes.io/projected/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-kube-api-access-g6fdj\") pod \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\" (UID: \"c1399892-8ce0-410d-9bd8-d1a2a40df2e8\") "
Nov 03 22:58:04.837460 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.837435    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-plugins-config-volume" pod "c1399892-8ce0-410d-9bd8-d1a2a40df2e8" (UID: "c1399892-8ce0-410d-9bd8-d1a2a40df2e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:04.837523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.837508    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twqhd\" (UniqueName: \"kubernetes.io/projected/9f615715-b0f7-4754-b079-f998dd0cf49d-kube-api-access-twqhd\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:04.839934 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.839908    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-kube-api-access-g6fdj" pod "c1399892-8ce0-410d-9bd8-d1a2a40df2e8" (UID: "c1399892-8ce0-410d-9bd8-d1a2a40df2e8"). InnerVolumeSpecName "kube-api-access-g6fdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:04.881541 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.881457    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"]
Nov 03 22:58:04.882566 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.882540    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a9ab23c4-serving-cert\""
Nov 03 22:58:04.939215 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.939173    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-metrics-certs\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:04.939385 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.939229    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:04.939475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.939428    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea694015-0f0b-494c-9855-9146c5b83c7f-openshift-service-ca\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:04.939581 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.939562    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gspkv\" (UniqueName: \"kubernetes.io/projected/ea694015-0f0b-494c-9855-9146c5b83c7f-kube-api-access-gspkv\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:04.939824 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.939798    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:04.939895 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:04.939832    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6fdj\" (UniqueName: \"kubernetes.io/projected/c1399892-8ce0-410d-9bd8-d1a2a40df2e8-kube-api-access-g6fdj\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:05.076823 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.076773    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea694015-0f0b-494c-9855-9146c5b83c7f-openshift-service-ca\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.078506 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.078479    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gspkv\" (UniqueName: \"kubernetes.io/projected/ea694015-0f0b-494c-9855-9146c5b83c7f-kube-api-access-gspkv\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.078874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.078843    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-metrics-certs\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.079037 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.079022    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.079347 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:05.079318    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-serving-cert: secret "maas-api-e2e-mt-a9ab23c4-serving-cert" not found
Nov 03 22:58:05.079514 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:05.079503    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls podName:ea694015-0f0b-494c-9855-9146c5b83c7f nodeName:}" failed. No retries permitted until 2026-11-03 22:58:05.57948158 +0000 UTC m=+2109.020018821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls") pod "maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j" (UID: "ea694015-0f0b-494c-9855-9146c5b83c7f") : secret "maas-api-e2e-mt-a9ab23c4-serving-cert" not found
Nov 03 22:58:05.079856 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.078231    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea694015-0f0b-494c-9855-9146c5b83c7f-openshift-service-ca\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.091061 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.091031    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-metrics-certs\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.142363 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.142280    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gspkv\" (UniqueName: \"kubernetes.io/projected/ea694015-0f0b-494c-9855-9146c5b83c7f-kube-api-access-gspkv\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.290604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.290571    5647 generic.go:356] "Generic (PLEG): container finished" podID="c1399892-8ce0-410d-9bd8-d1a2a40df2e8" containerID="a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846" exitCode=0
Nov 03 22:58:05.290821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.290634    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"
Nov 03 22:58:05.290821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.290679    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn" event={"ID":"c1399892-8ce0-410d-9bd8-d1a2a40df2e8","Type":"ContainerDied","Data":"a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846"}
Nov 03 22:58:05.290821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.290736    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn" event={"ID":"c1399892-8ce0-410d-9bd8-d1a2a40df2e8","Type":"ContainerDied","Data":"75d238751252a0215f5df7862b957d8c3a3a627b93c221333f4ffbd88133be15"}
Nov 03 22:58:05.290821 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.290761    5647 scope.go:122] "RemoveContainer" containerID="a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846"
Nov 03 22:58:05.292618 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.292596    5647 generic.go:356] "Generic (PLEG): container finished" podID="9f615715-b0f7-4754-b079-f998dd0cf49d" containerID="5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f" exitCode=0
Nov 03 22:58:05.292775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.292630    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq" event={"ID":"9f615715-b0f7-4754-b079-f998dd0cf49d","Type":"ContainerDied","Data":"5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f"}
Nov 03 22:58:05.292775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.292710    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"
Nov 03 22:58:05.292775 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.292728    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq" event={"ID":"9f615715-b0f7-4754-b079-f998dd0cf49d","Type":"ContainerDied","Data":"2d245c7b9a1aa89cf9a944d162535081d8b611e9efdf516fdc231c75901ff9c3"}
Nov 03 22:58:05.303544 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.303529    5647 scope.go:122] "RemoveContainer" containerID="a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846"
Nov 03 22:58:05.303845 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:05.303825    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846\": container with ID starting with a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846 not found: ID does not exist" containerID="a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846"
Nov 03 22:58:05.303900 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.303852    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846"} err="failed to get container status \"a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846\": rpc error: code = NotFound desc = could not find container \"a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846\": container with ID starting with a79570b7cecfd5ba99d3658de2e9f55d3bfe4fdd226ef921a1260e2b0d40c846 not found: ID does not exist"
Nov 03 22:58:05.303900 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.303869    5647 scope.go:122] "RemoveContainer" containerID="5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f"
Nov 03 22:58:05.313856 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.313832    5647 scope.go:122] "RemoveContainer" containerID="5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f"
Nov 03 22:58:05.314113 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:05.314092    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f\": container with ID starting with 5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f not found: ID does not exist" containerID="5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f"
Nov 03 22:58:05.314164 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.314120    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f"} err="failed to get container status \"5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f\": rpc error: code = NotFound desc = could not find container \"5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f\": container with ID starting with 5a8449488fcd365aa000efa155f98cb90c79ca6ea85e5730a5332c12da21565f not found: ID does not exist"
Nov 03 22:58:05.460716 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.460686    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"]
Nov 03 22:58:05.467605 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.467569    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq"]
Nov 03 22:58:05.587727 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.587654    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.590390 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.590358    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls\") pod \"maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:05.593757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.593733    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"]
Nov 03 22:58:05.598138 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.598114    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn"]
Nov 03 22:58:05.745893 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:05.745802    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:06.179614 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:06.179588    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"]
Nov 03 22:58:06.180869 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:58:06.180842    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea694015_0f0b_494c_9855_9146c5b83c7f.slice/crio-8b4b2c2e8305196f49270c90975cb7b570ff3fa946c3b0707aa922541ea68469 WatchSource:0}: Error finding container 8b4b2c2e8305196f49270c90975cb7b570ff3fa946c3b0707aa922541ea68469: Status 404 returned error can't find the container with id 8b4b2c2e8305196f49270c90975cb7b570ff3fa946c3b0707aa922541ea68469
Nov 03 22:58:06.187020 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:06.187000    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 22:58:06.302255 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:06.302222    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j" event={"ID":"ea694015-0f0b-494c-9855-9146c5b83c7f","Type":"ContainerStarted","Data":"8b4b2c2e8305196f49270c90975cb7b570ff3fa946c3b0707aa922541ea68469"}
Nov 03 22:58:06.698930 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:06.698897    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f615715-b0f7-4754-b079-f998dd0cf49d" path="/var/lib/kubelet/pods/9f615715-b0f7-4754-b079-f998dd0cf49d/volumes"
Nov 03 22:58:06.699379 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:06.699357    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1399892-8ce0-410d-9bd8-d1a2a40df2e8" path="/var/lib/kubelet/pods/c1399892-8ce0-410d-9bd8-d1a2a40df2e8/volumes"
Nov 03 22:58:07.312100 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:07.312068    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j" event={"ID":"ea694015-0f0b-494c-9855-9146c5b83c7f","Type":"ContainerStarted","Data":"f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76"}
Nov 03 22:58:07.312286 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:07.312133    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:07.432961 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:07.432823    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j" podStartSLOduration=2.524411119 podStartE2EDuration="3.432807645s" totalImagesPullingTime="908.396526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:58:07.429692435 +0000 UTC m=+2110.870229689" watchObservedRunningTime="2026-11-03 22:58:07.432807645 +0000 UTC m=+2110.873344883"
Nov 03 22:58:13.396281 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:13.396248    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 22:58:15.090999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:15.090960    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"]
Nov 03 22:58:15.140587 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:15.140550    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"]
Nov 03 22:58:16.044729 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:16.044688    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:58:17.123040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.122907    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"]
Nov 03 22:58:17.129496 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.129469    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:17.180526 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.180495    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"]
Nov 03 22:58:17.319693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.298325    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-924fba\""
Nov 03 22:58:17.320273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.318096    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-924fba-dockercfg-wrt55\""
Nov 03 22:58:17.409754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.406045    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmb76\" (UniqueName: \"kubernetes.io/projected/b834e991-6ce5-4d77-84ca-b8f3a2225a84-kube-api-access-dmb76\") pod \"payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7\" (UID: \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:17.409754 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.408711    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b834e991-6ce5-4d77-84ca-b8f3a2225a84-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7\" (UID: \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:17.438699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.436490    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"]
Nov 03 22:58:17.451167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.451132    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"]
Nov 03 22:58:17.451369 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.451354    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:17.510353 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.510318    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgt4t\" (UniqueName: \"kubernetes.io/projected/0417d2a7-b9ce-484c-89af-c7a80319b260-kube-api-access-lgt4t\") pod \"payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6\" (UID: \"0417d2a7-b9ce-484c-89af-c7a80319b260\") " pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:17.510568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.510399    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmb76\" (UniqueName: \"kubernetes.io/projected/b834e991-6ce5-4d77-84ca-b8f3a2225a84-kube-api-access-dmb76\") pod \"payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7\" (UID: \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:17.510568 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.510545    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0417d2a7-b9ce-484c-89af-c7a80319b260-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6\" (UID: \"0417d2a7-b9ce-484c-89af-c7a80319b260\") " pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:17.510742 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.510727    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b834e991-6ce5-4d77-84ca-b8f3a2225a84-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7\" (UID: \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:17.511401 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.511384    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b834e991-6ce5-4d77-84ca-b8f3a2225a84-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7\" (UID: \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:17.573580 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.573537    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmb76\" (UniqueName: \"kubernetes.io/projected/b834e991-6ce5-4d77-84ca-b8f3a2225a84-kube-api-access-dmb76\") pod \"payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7\" (UID: \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:17.612388 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.612348    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgt4t\" (UniqueName: \"kubernetes.io/projected/0417d2a7-b9ce-484c-89af-c7a80319b260-kube-api-access-lgt4t\") pod \"payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6\" (UID: \"0417d2a7-b9ce-484c-89af-c7a80319b260\") " pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:17.612561 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.612477    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0417d2a7-b9ce-484c-89af-c7a80319b260-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6\" (UID: \"0417d2a7-b9ce-484c-89af-c7a80319b260\") " pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:17.613216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.613195    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0417d2a7-b9ce-484c-89af-c7a80319b260-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6\" (UID: \"0417d2a7-b9ce-484c-89af-c7a80319b260\") " pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:17.681357 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.681276    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgt4t\" (UniqueName: \"kubernetes.io/projected/0417d2a7-b9ce-484c-89af-c7a80319b260-kube-api-access-lgt4t\") pod \"payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6\" (UID: \"0417d2a7-b9ce-484c-89af-c7a80319b260\") " pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:17.754639 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.754603    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:17.768267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:17.768161    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:18.137733 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.137692    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"]
Nov 03 22:58:18.176838 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:58:18.176809    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb834e991_6ce5_4d77_84ca_b8f3a2225a84.slice/crio-b040f66050ab1321fcf152505287dae00d2ca68c38f51daba6f8e223b510c3fa WatchSource:0}: Error finding container b040f66050ab1321fcf152505287dae00d2ca68c38f51daba6f8e223b510c3fa: Status 404 returned error can't find the container with id b040f66050ab1321fcf152505287dae00d2ca68c38f51daba6f8e223b510c3fa
Nov 03 22:58:18.188087 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.188062    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"]
Nov 03 22:58:18.191363 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:58:18.191340    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0417d2a7_b9ce_484c_89af_c7a80319b260.slice/crio-4fb7087b776813d696783e65629ba3fec9d283d2538d54f33fdac6b91d8c4552 WatchSource:0}: Error finding container 4fb7087b776813d696783e65629ba3fec9d283d2538d54f33fdac6b91d8c4552: Status 404 returned error can't find the container with id 4fb7087b776813d696783e65629ba3fec9d283d2538d54f33fdac6b91d8c4552
Nov 03 22:58:18.520166 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.520121    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6" event={"ID":"0417d2a7-b9ce-484c-89af-c7a80319b260","Type":"ContainerStarted","Data":"a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458"}
Nov 03 22:58:18.520403 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.520232    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6" event={"ID":"0417d2a7-b9ce-484c-89af-c7a80319b260","Type":"ContainerStarted","Data":"4fb7087b776813d696783e65629ba3fec9d283d2538d54f33fdac6b91d8c4552"}
Nov 03 22:58:18.520403 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.520368    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:18.521877 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.521854    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7" event={"ID":"b834e991-6ce5-4d77-84ca-b8f3a2225a84","Type":"ContainerStarted","Data":"5c9bd9d55bc5dbac44325565a93f80b70b1254bc901db4447811162de5c9a7c7"}
Nov 03 22:58:18.522004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.521889    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7" event={"ID":"b834e991-6ce5-4d77-84ca-b8f3a2225a84","Type":"ContainerStarted","Data":"b040f66050ab1321fcf152505287dae00d2ca68c38f51daba6f8e223b510c3fa"}
Nov 03 22:58:18.522004 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.521929    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:18.636167 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:18.636101    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6" podStartSLOduration=1.6360841860000002 podStartE2EDuration="1.636084186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:58:18.63472618 +0000 UTC m=+2122.075263440" watchObservedRunningTime="2026-11-03 22:58:18.636084186 +0000 UTC m=+2122.076621449"
Nov 03 22:58:23.827471 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:23.827417    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7" podStartSLOduration=6.827404252 podStartE2EDuration="6.827404252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:58:18.756716905 +0000 UTC m=+2122.197254162" watchObservedRunningTime="2026-11-03 22:58:23.827404252 +0000 UTC m=+2127.267941510"
Nov 03 22:58:23.830374 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:23.830345    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"]
Nov 03 22:58:23.946759 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:23.946724    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"]
Nov 03 22:58:28.842773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:28.842724    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8" podUID="84e7a354-3e83-4cec-93e1-50f034204374" containerName="payload-processing" containerID="cri-o://a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f" gracePeriod=25
Nov 03 22:58:28.947624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:28.947354    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg" podUID="b42582d3-f555-4fc5-8e82-732be9385712" containerName="payload-pre-processing" containerID="cri-o://5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e" gracePeriod=25
Nov 03 22:58:29.328776 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.328740    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:58:29.361907 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.361879    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:58:29.426842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.426746    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b42582d3-f555-4fc5-8e82-732be9385712-kube-api-access-lf6t6\" (UniqueName: \"kubernetes.io/projected/b42582d3-f555-4fc5-8e82-732be9385712-kube-api-access-lf6t6\") pod \"b42582d3-f555-4fc5-8e82-732be9385712\" (UID: \"b42582d3-f555-4fc5-8e82-732be9385712\") "
Nov 03 22:58:29.427009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.426856    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84e7a354-3e83-4cec-93e1-50f034204374-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84e7a354-3e83-4cec-93e1-50f034204374-plugins-config-volume\") pod \"84e7a354-3e83-4cec-93e1-50f034204374\" (UID: \"84e7a354-3e83-4cec-93e1-50f034204374\") "
Nov 03 22:58:29.427009 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.426942    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84e7a354-3e83-4cec-93e1-50f034204374-kube-api-access-5bglz\" (UniqueName: \"kubernetes.io/projected/84e7a354-3e83-4cec-93e1-50f034204374-kube-api-access-5bglz\") pod \"84e7a354-3e83-4cec-93e1-50f034204374\" (UID: \"84e7a354-3e83-4cec-93e1-50f034204374\") "
Nov 03 22:58:29.427137 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.427041    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b42582d3-f555-4fc5-8e82-732be9385712-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b42582d3-f555-4fc5-8e82-732be9385712-plugins-config-volume\") pod \"b42582d3-f555-4fc5-8e82-732be9385712\" (UID: \"b42582d3-f555-4fc5-8e82-732be9385712\") "
Nov 03 22:58:29.427367 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.427334    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e7a354-3e83-4cec-93e1-50f034204374-plugins-config-volume" pod "84e7a354-3e83-4cec-93e1-50f034204374" (UID: "84e7a354-3e83-4cec-93e1-50f034204374"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:29.427525 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.427498    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b42582d3-f555-4fc5-8e82-732be9385712-plugins-config-volume" pod "b42582d3-f555-4fc5-8e82-732be9385712" (UID: "b42582d3-f555-4fc5-8e82-732be9385712"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:29.427814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.427784    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84e7a354-3e83-4cec-93e1-50f034204374-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:29.427814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.427810    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b42582d3-f555-4fc5-8e82-732be9385712-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:29.429848 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.429813    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e7a354-3e83-4cec-93e1-50f034204374-kube-api-access-5bglz" pod "84e7a354-3e83-4cec-93e1-50f034204374" (UID: "84e7a354-3e83-4cec-93e1-50f034204374"). InnerVolumeSpecName "kube-api-access-5bglz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:29.429937 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.429847    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b42582d3-f555-4fc5-8e82-732be9385712-kube-api-access-lf6t6" pod "b42582d3-f555-4fc5-8e82-732be9385712" (UID: "b42582d3-f555-4fc5-8e82-732be9385712"). InnerVolumeSpecName "kube-api-access-lf6t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:29.529479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.529437    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf6t6\" (UniqueName: \"kubernetes.io/projected/b42582d3-f555-4fc5-8e82-732be9385712-kube-api-access-lf6t6\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:29.529479 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.529476    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bglz\" (UniqueName: \"kubernetes.io/projected/84e7a354-3e83-4cec-93e1-50f034204374-kube-api-access-5bglz\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:29.532010 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.531977    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 22:58:29.532116 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.532053    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 22:58:29.617509 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.617479    5647 generic.go:356] "Generic (PLEG): container finished" podID="b42582d3-f555-4fc5-8e82-732be9385712" containerID="5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e" exitCode=0
Nov 03 22:58:29.617745 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.617526    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"
Nov 03 22:58:29.617745 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.617564    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg" event={"ID":"b42582d3-f555-4fc5-8e82-732be9385712","Type":"ContainerDied","Data":"5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e"}
Nov 03 22:58:29.617745 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.617620    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg" event={"ID":"b42582d3-f555-4fc5-8e82-732be9385712","Type":"ContainerDied","Data":"d395433bb7d98c611c4de104517e9ddfbeb5c20eae763393aa3087eb01aa62bb"}
Nov 03 22:58:29.617745 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.617651    5647 scope.go:122] "RemoveContainer" containerID="5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e"
Nov 03 22:58:29.619165 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.619141    5647 generic.go:356] "Generic (PLEG): container finished" podID="84e7a354-3e83-4cec-93e1-50f034204374" containerID="a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f" exitCode=0
Nov 03 22:58:29.619273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.619193    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"
Nov 03 22:58:29.619273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.619211    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8" event={"ID":"84e7a354-3e83-4cec-93e1-50f034204374","Type":"ContainerDied","Data":"a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f"}
Nov 03 22:58:29.619273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.619250    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8" event={"ID":"84e7a354-3e83-4cec-93e1-50f034204374","Type":"ContainerDied","Data":"4e03185585cbad2ceb89de39185ce4f329a2985be3747ca5f65a790cc2e0b477"}
Nov 03 22:58:29.630303 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.630284    5647 scope.go:122] "RemoveContainer" containerID="5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e"
Nov 03 22:58:29.630613 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:29.630584    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e\": container with ID starting with 5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e not found: ID does not exist" containerID="5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e"
Nov 03 22:58:29.630725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.630621    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e"} err="failed to get container status \"5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e\": rpc error: code = NotFound desc = could not find container \"5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e\": container with ID starting with 5df821f63aa5bda272c66a984e29503d8d73ccc0999b75f299ea3f7f9f76c42e not found: ID does not exist"
Nov 03 22:58:29.630725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.630637    5647 scope.go:122] "RemoveContainer" containerID="a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f"
Nov 03 22:58:29.640600 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.640583    5647 scope.go:122] "RemoveContainer" containerID="a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f"
Nov 03 22:58:29.640893 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:29.640874    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f\": container with ID starting with a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f not found: ID does not exist" containerID="a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f"
Nov 03 22:58:29.640969 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.640899    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f"} err="failed to get container status \"a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f\": rpc error: code = NotFound desc = could not find container \"a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f\": container with ID starting with a6bd56c856aa7316ed984c0cdf0146f9c0c169ef3adecc5cd627ce0da6bb698f not found: ID does not exist"
Nov 03 22:58:29.908006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.907971    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"]
Nov 03 22:58:29.912371 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:29.912343    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg"]
Nov 03 22:58:30.020693 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.020583    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"]
Nov 03 22:58:30.024366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.024339    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8"]
Nov 03 22:58:30.114554 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.114491    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main" containerID="cri-o://38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d" gracePeriod=45
Nov 03 22:58:30.163957 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.163868    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main" containerID="cri-o://d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78" gracePeriod=45
Nov 03 22:58:30.480891 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.480854    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:58:30.484254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.484233    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:58:30.545120 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545088    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-model-cache\") pod \"0a7bd6ca-2661-493c-85d2-b2c490763360\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") "
Nov 03 22:58:30.545302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545134    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tls-certs\") pod \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") "
Nov 03 22:58:30.545302 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545198    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a7bd6ca-2661-493c-85d2-b2c490763360-kube-api-access-t6d8w\" (UniqueName: \"kubernetes.io/projected/0a7bd6ca-2661-493c-85d2-b2c490763360-kube-api-access-t6d8w\") pod \"0a7bd6ca-2661-493c-85d2-b2c490763360\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") "
Nov 03 22:58:30.545448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545332    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-home\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-home\") pod \"0a7bd6ca-2661-493c-85d2-b2c490763360\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") "
Nov 03 22:58:30.545448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545367    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-dshm\") pod \"0a7bd6ca-2661-493c-85d2-b2c490763360\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") "
Nov 03 22:58:30.545448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545369    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-model-cache" pod "0a7bd6ca-2661-493c-85d2-b2c490763360" (UID: "0a7bd6ca-2661-493c-85d2-b2c490763360"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:30.545448 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545418    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tmp-dir\") pod \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") "
Nov 03 22:58:30.545658 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545481    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-dshm\") pod \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") "
Nov 03 22:58:30.545658 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545543    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a7bd6ca-2661-493c-85d2-b2c490763360-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a7bd6ca-2661-493c-85d2-b2c490763360-tls-certs\") pod \"0a7bd6ca-2661-493c-85d2-b2c490763360\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") "
Nov 03 22:58:30.545658 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545576    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-home" pod "0a7bd6ca-2661-493c-85d2-b2c490763360" (UID: "0a7bd6ca-2661-493c-85d2-b2c490763360"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:30.545658 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545600    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-tmp-dir\") pod \"0a7bd6ca-2661-493c-85d2-b2c490763360\" (UID: \"0a7bd6ca-2661-493c-85d2-b2c490763360\") "
Nov 03 22:58:30.545888 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545722    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tmp-dir" pod "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" (UID: "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:30.545888 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545742    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-kube-api-access-c46dp\" (UniqueName: \"kubernetes.io/projected/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-kube-api-access-c46dp\") pod \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") "
Nov 03 22:58:30.545888 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545776    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-model-cache\") pod \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") "
Nov 03 22:58:30.545888 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545807    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-home\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-home\") pod \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\" (UID: \"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2\") "
Nov 03 22:58:30.546084 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.545920    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-tmp-dir" pod "0a7bd6ca-2661-493c-85d2-b2c490763360" (UID: "0a7bd6ca-2661-493c-85d2-b2c490763360"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:30.546547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.546218    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-model-cache" pod "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" (UID: "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:30.546547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.546483    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.546547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.546441    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-home" pod "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" (UID: "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:30.546864 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.546504    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.546864 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.546690    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.546864 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.546706    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.546864 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.546723    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.547945 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.547921    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tls-certs" pod "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" (UID: "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:58:30.548116 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.548101    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7bd6ca-2661-493c-85d2-b2c490763360-kube-api-access-t6d8w" pod "0a7bd6ca-2661-493c-85d2-b2c490763360" (UID: "0a7bd6ca-2661-493c-85d2-b2c490763360"). InnerVolumeSpecName "kube-api-access-t6d8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:30.548250 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.548232    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-dshm" pod "0a7bd6ca-2661-493c-85d2-b2c490763360" (UID: "0a7bd6ca-2661-493c-85d2-b2c490763360"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:30.548296 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.548241    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a7bd6ca-2661-493c-85d2-b2c490763360-tls-certs" pod "0a7bd6ca-2661-493c-85d2-b2c490763360" (UID: "0a7bd6ca-2661-493c-85d2-b2c490763360"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:58:30.548335 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.548323    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-dshm" pod "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" (UID: "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:30.548523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.548498    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-kube-api-access-c46dp" pod "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" (UID: "6c82ed1b-3e6e-48eb-986b-0285d98d7eb2"). InnerVolumeSpecName "kube-api-access-c46dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:30.639044 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.638921    5647 generic.go:356] "Generic (PLEG): container finished" podID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerID="d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78" exitCode=0
Nov 03 22:58:30.640395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.639498    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" event={"ID":"0a7bd6ca-2661-493c-85d2-b2c490763360","Type":"ContainerDied","Data":"d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78"}
Nov 03 22:58:30.640395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.639552    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469" event={"ID":"0a7bd6ca-2661-493c-85d2-b2c490763360","Type":"ContainerDied","Data":"a8b478d662b484e94a061ebe63da242d8dd819a8b548600888c312fa30114623"}
Nov 03 22:58:30.640395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.639577    5647 scope.go:122] "RemoveContainer" containerID="d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78"
Nov 03 22:58:30.640395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.639749    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"
Nov 03 22:58:30.647112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.646967    5647 generic.go:356] "Generic (PLEG): container finished" podID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerID="38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d" exitCode=0
Nov 03 22:58:30.647112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.647044    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" event={"ID":"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2","Type":"ContainerDied","Data":"38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d"}
Nov 03 22:58:30.647112 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.647080    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"
Nov 03 22:58:30.647449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.647388    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl" event={"ID":"6c82ed1b-3e6e-48eb-986b-0285d98d7eb2","Type":"ContainerDied","Data":"8b32326cf7fcc132a105a8f580cdea6367a83fd7253838904736bff0ced75887"}
Nov 03 22:58:30.648704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.648646    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.648813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.648706    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a7bd6ca-2661-493c-85d2-b2c490763360-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.648813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.648720    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c46dp\" (UniqueName: \"kubernetes.io/projected/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-kube-api-access-c46dp\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.648813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.648729    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.648813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.648738    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.648813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.648746    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6d8w\" (UniqueName: \"kubernetes.io/projected/0a7bd6ca-2661-493c-85d2-b2c490763360-kube-api-access-t6d8w\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.648813 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.648754    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a7bd6ca-2661-493c-85d2-b2c490763360-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:30.664230 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.664122    5647 scope.go:122] "RemoveContainer" containerID="d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78"
Nov 03 22:58:30.664891 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:30.664867    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78\": container with ID starting with d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78 not found: ID does not exist" containerID="d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78"
Nov 03 22:58:30.665007 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.664902    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78"} err="failed to get container status \"d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78\": rpc error: code = NotFound desc = could not find container \"d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78\": container with ID starting with d5e264f13efd440f42fefa74e2d0f45ebc15329222ceed2c37281f337a078d78 not found: ID does not exist"
Nov 03 22:58:30.665007 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.664926    5647 scope.go:122] "RemoveContainer" containerID="38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d"
Nov 03 22:58:30.675018 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.675000    5647 scope.go:122] "RemoveContainer" containerID="38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d"
Nov 03 22:58:30.675299 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:30.675278    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d\": container with ID starting with 38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d not found: ID does not exist" containerID="38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d"
Nov 03 22:58:30.675374 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.675319    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d"} err="failed to get container status \"38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d\": rpc error: code = NotFound desc = could not find container \"38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d\": container with ID starting with 38fbde91b7e86c1c37051f0d2ebf1f5d0dc47e85b55c55cac946019249f6bf9d not found: ID does not exist"
Nov 03 22:58:30.697857 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.697798    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84e7a354-3e83-4cec-93e1-50f034204374" path="/var/lib/kubelet/pods/84e7a354-3e83-4cec-93e1-50f034204374/volumes"
Nov 03 22:58:30.698271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.698251    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b42582d3-f555-4fc5-8e82-732be9385712" path="/var/lib/kubelet/pods/b42582d3-f555-4fc5-8e82-732be9385712/volumes"
Nov 03 22:58:30.844967 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.844932    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"]
Nov 03 22:58:30.848214 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.848192    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/premium-w5-cbd26c-kserve-84d568866f-6z469"]
Nov 03 22:58:30.965617 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:30.965573    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"]
Nov 03 22:58:31.040965 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:31.038825    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-cbd26c/distinct-2-w5-cbd26c-kserve-5d96dbb5c5-xsskl"]
Nov 03 22:58:32.697522 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:32.697483    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" path="/var/lib/kubelet/pods/0a7bd6ca-2661-493c-85d2-b2c490763360/volumes"
Nov 03 22:58:32.698032 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:32.698016    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" path="/var/lib/kubelet/pods/6c82ed1b-3e6e-48eb-986b-0285d98d7eb2/volumes"
Nov 03 22:58:35.759782 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:35.759344    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:58:40.327535 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:40.327501    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"]
Nov 03 22:58:40.328023 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:40.327708    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" containerID="cri-o://9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866" gracePeriod=30
Nov 03 22:58:40.907955 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:40.907920    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"]
Nov 03 22:58:40.908153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:40.908116    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" containerID="cri-o://e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf" gracePeriod=30
Nov 03 22:58:41.632842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:41.632803    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"]
Nov 03 22:58:41.633219 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:41.633056    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy" containerID="cri-o://e2431110e7b1da9defe3bdc6a3858f2f8b77f66667da8f835aeb41ffaf02b2c8" gracePeriod=30
Nov 03 22:58:43.170506 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:43.170471    5647 patch_prober.go:29] interesting pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:58:43.170899 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:43.170539    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:58:43.450363 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:43.450287    5647 patch_prober.go:29] interesting pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:58:43.450363 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:43.450347    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:58:44.806519 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:44.806479    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 22:58:44.807023 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:44.806548    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 22:58:45.632861 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.632835    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:58:45.783149 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783117    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-credential-socket\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783177    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-certs\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783222    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-socket\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783264    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istiod-ca-cert\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783292    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-kube-api-access-7vjf8\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-kube-api-access-7vjf8\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783332 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783315    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-envoy\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783421    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-data\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783419    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-credential-socket" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:45.783604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783528    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-certs" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:45.783604 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783594    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-token\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-token\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783884 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783610    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-socket" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:45.783884 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783703    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-podinfo\") pod \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\" (UID: \"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec\") "
Nov 03 22:58:45.783884 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783757    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-data" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:45.783884 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.783828    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istiod-ca-cert" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:45.784395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.784308    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.784395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.784344    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.784395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.784357    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.784395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.784365    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.784395 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.784374    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.785993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.785974    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-podinfo" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:58:45.786068 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.785991    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-kube-api-access-7vjf8" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "kube-api-access-7vjf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:45.786162 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.786146    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-token" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:45.786286 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.786268    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-envoy" pod "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" (UID: "7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:45.831787 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.831750    5647 generic.go:356] "Generic (PLEG): container finished" podID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerID="9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866" exitCode=0
Nov 03 22:58:45.832211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.831798    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"
Nov 03 22:58:45.832211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.831836    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" event={"ID":"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec","Type":"ContainerDied","Data":"9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866"}
Nov 03 22:58:45.832211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.831896    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql" event={"ID":"7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec","Type":"ContainerDied","Data":"090afd9266ec04563daac5ded7455b15a5c789b1fafeba77e200ac471647687b"}
Nov 03 22:58:45.832211 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.831924    5647 scope.go:122] "RemoveContainer" containerID="9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866"
Nov 03 22:58:45.844309 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.844275    5647 scope.go:122] "RemoveContainer" containerID="9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866"
Nov 03 22:58:45.844613 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:45.844588    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866\": container with ID starting with 9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866 not found: ID does not exist" containerID="9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866"
Nov 03 22:58:45.844781 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.844624    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866"} err="failed to get container status \"9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866\": rpc error: code = NotFound desc = could not find container \"9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866\": container with ID starting with 9b4e1db9cc616724109fe6a7cd02edbd4b76d1e9b23741c3f2bc31ef79510866 not found: ID does not exist"
Nov 03 22:58:45.885491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.885395    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vjf8\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-kube-api-access-7vjf8\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.885491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.885427    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.885491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.885438    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.885491 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.885447    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:45.997096 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:45.997068    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"]
Nov 03 22:58:46.001103 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.001078    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql"]
Nov 03 22:58:46.047706 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:46.047082    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:58:46.160973 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.160949    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:58:46.291449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291414    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-envoy\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.291449 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291454    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-credential-socket\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.291771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291473    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-socket\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.291771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291496    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-certs\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.291771 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291548    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istiod-ca-cert\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.292129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291788    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-kube-api-access-gbmqm\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-kube-api-access-gbmqm\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.292129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291828    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-credential-socket" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:46.292129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291870    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-socket" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:46.292129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291925    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-certs" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:46.292129 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.291962    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istiod-ca-cert" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:46.292408 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.292207    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-podinfo\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.292408 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.292260    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-token\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-token\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.292408 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.292318    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-data\") pod \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\" (UID: \"7f0b763c-b2c0-410b-bcf2-b7728d61bf55\") "
Nov 03 22:58:46.292880 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.292842    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-data" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:46.293290 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.293260    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.293390 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.293293    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.293390 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.293309    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.293390 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.293324    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.293390 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.293338    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.294496 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.294473    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-envoy" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:46.294594 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.294528    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-token" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:46.294594 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.294565    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-kube-api-access-gbmqm" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "kube-api-access-gbmqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:46.295157 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.295140    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-podinfo" pod "7f0b763c-b2c0-410b-bcf2-b7728d61bf55" (UID: "7f0b763c-b2c0-410b-bcf2-b7728d61bf55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:58:46.399807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.395148    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbmqm\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-kube-api-access-gbmqm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.399807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.395190    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.399807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.395206    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.399807 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.395218    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f0b763c-b2c0-410b-bcf2-b7728d61bf55-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:46.699503 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.699466    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" path="/var/lib/kubelet/pods/7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec/volumes"
Nov 03 22:58:46.841534 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.841488    5647 generic.go:356] "Generic (PLEG): container finished" podID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerID="e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf" exitCode=0
Nov 03 22:58:46.842244 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.841737    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" event={"ID":"7f0b763c-b2c0-410b-bcf2-b7728d61bf55","Type":"ContainerDied","Data":"e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf"}
Nov 03 22:58:46.842244 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.841799    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"
Nov 03 22:58:46.842244 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.841816    5647 scope.go:122] "RemoveContainer" containerID="e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf"
Nov 03 22:58:46.842244 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.841801    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf" event={"ID":"7f0b763c-b2c0-410b-bcf2-b7728d61bf55","Type":"ContainerDied","Data":"feaa75f0f383fcc20916ad9973b4fddaeff149e614273a58af7de73510562799"}
Nov 03 22:58:46.847638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.847611    5647 generic.go:356] "Generic (PLEG): container finished" podID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerID="e2431110e7b1da9defe3bdc6a3858f2f8b77f66667da8f835aeb41ffaf02b2c8" exitCode=0
Nov 03 22:58:46.847791 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.847656    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" event={"ID":"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa","Type":"ContainerDied","Data":"e2431110e7b1da9defe3bdc6a3858f2f8b77f66667da8f835aeb41ffaf02b2c8"}
Nov 03 22:58:46.861778 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.861705    5647 scope.go:122] "RemoveContainer" containerID="e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf"
Nov 03 22:58:46.862136 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:46.862067    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf\": container with ID starting with e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf not found: ID does not exist" containerID="e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf"
Nov 03 22:58:46.862136 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.862106    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf"} err="failed to get container status \"e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf\": rpc error: code = NotFound desc = could not find container \"e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf\": container with ID starting with e80dda42688b109b5ab0e36b771e72e35fa10418fddffcece628601261da6eaf not found: ID does not exist"
Nov 03 22:58:46.923758 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:46.923734    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:58:47.006874 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.006840    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"]
Nov 03 22:58:47.055547 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.055516    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf"]
Nov 03 22:58:47.104212 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104182    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istiod-ca-cert\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104228    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-credential-socket\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104268    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-token\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-token\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104305    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-certs\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104352    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-podinfo\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104391 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104385    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-kube-api-access-6cg82\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-kube-api-access-6cg82\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104443    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-data\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104490    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-socket\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104534    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-envoy\") pod \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\" (UID: \"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa\") "
Nov 03 22:58:47.104706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104568    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istiod-ca-cert" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:47.104706 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104575    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-credential-socket" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:47.104984 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104782    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-certs" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:47.104984 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104912    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-data" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:47.105254 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.104998    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-socket" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:47.105613 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.105587    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.105730 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.105618    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.105730 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.105635    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.105730 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.105650    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.105730 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.105681    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.106799 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.106775    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-token" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:47.106890 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.106805    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-kube-api-access-6cg82" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "kube-api-access-6cg82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:47.106890 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.106825    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-podinfo" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:58:47.106959 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.106953    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-envoy" pod "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" (UID: "d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:58:47.207643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.207608    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cg82\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-kube-api-access-6cg82\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.207643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.207638    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.207643 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.207649    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.207962 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.207657    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:47.857733 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.857684    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q" event={"ID":"d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa","Type":"ContainerDied","Data":"a7675714b5c1d5d474b5c7ef058176a17e507466cb1904933544ce86d12c52ec"}
Nov 03 22:58:47.858237 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.857749    5647 scope.go:122] "RemoveContainer" containerID="e2431110e7b1da9defe3bdc6a3858f2f8b77f66667da8f835aeb41ffaf02b2c8"
Nov 03 22:58:47.858237 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:47.857698    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"
Nov 03 22:58:48.033312 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:48.033281    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"]
Nov 03 22:58:48.082704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:48.082660    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q"]
Nov 03 22:58:48.697089 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:48.697055    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" path="/var/lib/kubelet/pods/7f0b763c-b2c0-410b-bcf2-b7728d61bf55/volumes"
Nov 03 22:58:48.697561 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:48.697546    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" path="/var/lib/kubelet/pods/d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa/volumes"
Nov 03 22:58:54.481859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.481826    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"]
Nov 03 22:58:54.482326 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.482052    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx" podUID="fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" containerName="maas-api" containerID="cri-o://b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d" gracePeriod=30
Nov 03 22:58:54.821084 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.821054    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:58:54.893773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.892108    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls\") pod \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") "
Nov 03 22:58:54.893773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.892314    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-metrics-certs\") pod \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") "
Nov 03 22:58:54.893773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.892359    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-openshift-service-ca\") pod \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") "
Nov 03 22:58:54.893773 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.892420    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-kube-api-access-fslh7\" (UniqueName: \"kubernetes.io/projected/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-kube-api-access-fslh7\") pod \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\" (UID: \"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833\") "
Nov 03 22:58:54.919694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.918687    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-openshift-service-ca" pod "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" (UID: "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:58:54.920349 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.920174    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls" pod "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" (UID: "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:58:54.941707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.939789    5647 generic.go:356] "Generic (PLEG): container finished" podID="fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" containerID="b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d" exitCode=0
Nov 03 22:58:54.941707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.939881    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx" event={"ID":"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833","Type":"ContainerDied","Data":"b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d"}
Nov 03 22:58:54.941707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.939934    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx" event={"ID":"fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833","Type":"ContainerDied","Data":"be5b48ed79bebe4efa957353eced4341d40331c57d06e0933d4336d80663f75d"}
Nov 03 22:58:54.941707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.939964    5647 scope.go:122] "RemoveContainer" containerID="b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d"
Nov 03 22:58:54.941707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.940131    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"
Nov 03 22:58:54.976707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.972949    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-kube-api-access-fslh7" pod "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" (UID: "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833"). InnerVolumeSpecName "kube-api-access-fslh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:58:54.976707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:54.973043    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-metrics-certs" pod "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" (UID: "fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 22:58:55.012125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:55.012089    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:55.012125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:55.012126    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:55.012416 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:55.012140    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fslh7\" (UniqueName: \"kubernetes.io/projected/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-kube-api-access-fslh7\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:55.012416 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:55.012153    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:58:55.023842 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:55.023815    5647 scope.go:122] "RemoveContainer" containerID="b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d"
Nov 03 22:58:55.024251 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:58:55.024229    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d\": container with ID starting with b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d not found: ID does not exist" containerID="b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d"
Nov 03 22:58:55.024344 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:55.024265    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d"} err="failed to get container status \"b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d\": rpc error: code = NotFound desc = could not find container \"b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d\": container with ID starting with b54e962d1248d7e1ae0b63bf110e95a3b7091e1f91e4459120f4df48fe4fdb6d not found: ID does not exist"
Nov 03 22:58:55.360728 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:55.360694    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"]
Nov 03 22:58:55.364871 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:55.364845    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-0406a8-7fcd48d458-krzfx"]
Nov 03 22:58:56.701878 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:58:56.701840    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" path="/var/lib/kubelet/pods/fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833/volumes"
Nov 03 22:59:10.947131 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.947094    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"]
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948094    5647 cpu_manager.go:414] "containerMap: removing container" podUID="b42582d3-f555-4fc5-8e82-732be9385712" containerName="payload-pre-processing"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948113    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b42582d3-f555-4fc5-8e82-732be9385712" containerName="payload-pre-processing"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948127    5647 cpu_manager.go:414] "containerMap: removing container" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948133    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948160    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948166    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948189    5647 cpu_manager.go:414] "containerMap: removing container" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948194    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948202    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948206    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948213    5647 cpu_manager.go:414] "containerMap: removing container" podUID="fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" containerName="maas-api"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948218    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" containerName="maas-api"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948227    5647 cpu_manager.go:414] "containerMap: removing container" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948231    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948248    5647 cpu_manager.go:414] "containerMap: removing container" podUID="84e7a354-3e83-4cec-93e1-50f034204374" containerName="payload-processing"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948252    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84e7a354-3e83-4cec-93e1-50f034204374" containerName="payload-processing"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948351    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="b42582d3-f555-4fc5-8e82-732be9385712" containerName="payload-pre-processing"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948358    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0b06c2b-0aad-497c-a4e9-9d4c5751bdfa" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948371    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f0b763c-b2c0-410b-bcf2-b7728d61bf55" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948379    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c82ed1b-3e6e-48eb-986b-0285d98d7eb2" containerName="main"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948386    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e0a1cd5-1e6a-4efc-ae19-50d4c4ea07ec" containerName="istio-proxy"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948390    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdc1f1d1-8fb2-4e15-853f-2bbad5fa9833" containerName="maas-api"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948397    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a7bd6ca-2661-493c-85d2-b2c490763360" containerName="main"
Nov 03 22:59:10.949564 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.948410    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="84e7a354-3e83-4cec-93e1-50f034204374" containerName="payload-processing"
Nov 03 22:59:10.952725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.952702    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:10.996191 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:10.996158    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"]
Nov 03 22:59:11.000523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.000505    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.021245 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021217    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.021406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021253    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rqrf\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-kube-api-access-5rqrf\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.021406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021273    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.021406 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021338    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.021581 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021441    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.021581 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021504    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a163038-fb12-44bf-9948-af32bee1357e-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.021711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021578    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a163038-fb12-44bf-9948-af32bee1357e-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.021780 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021737    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.021834 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.021804    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.045911 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.045879    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"]
Nov 03 22:59:11.095583 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.095549    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"]
Nov 03 22:59:11.123486 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.123442    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.123725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.123590    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x78xx\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-kube-api-access-x78xx\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.123725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.123629    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.123725 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.123658    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.123911 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.123815    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.123911 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.123859    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.124014 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.123929    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rqrf\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-kube-api-access-5rqrf\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.124293 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.124267    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.124293 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.123968    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.124475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.124360    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.124475 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.124456    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.124586 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.124490    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.124836 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.124815    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a163038-fb12-44bf-9948-af32bee1357e-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.124991 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.124966    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.125078 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.124851    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.125232 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.125210    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.125402 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.125372    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.125707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.125414    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a163038-fb12-44bf-9948-af32bee1357e-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.125707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.125487    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.125707 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.125514    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.126271 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.126244    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a163038-fb12-44bf-9948-af32bee1357e-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.126523 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.126499    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.126704 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.126682    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.127900 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.127874    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a163038-fb12-44bf-9948-af32bee1357e-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.129399 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.129368    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.185912 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.185846    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rqrf\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-kube-api-access-5rqrf\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.186459 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.186429    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.228132 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228102    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228152    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228190    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228283 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228243    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228402 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228337    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228458 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228401    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x78xx\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-kube-api-access-x78xx\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228512 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228489    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-certs\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228563 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228542    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-data\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228616 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228594    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228610    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228643    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.228814 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228752    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.229008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228982    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-credential-socket\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.229113 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.228983    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istiod-ca-cert\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.230881 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.230861    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-envoy\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.231197 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.231181    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-podinfo\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.267091 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.267071    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:11.289584 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.289554    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-token\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.303711 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.301248    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x78xx\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-kube-api-access-x78xx\") pod \"e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") " pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.315624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.315569    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:11.735339 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.735307    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"]
Nov 03 22:59:11.738137 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.738116    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"]
Nov 03 22:59:11.764813 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:59:11.764771    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d65809b_0adb_4d76_b9f5_ca690d30df9b.slice/crio-8e2338cf4385a6fdd9cfc62b8e55dea6a7d69bf24ffc5aa6dfc18e1e06119ef5 WatchSource:0}: Error finding container 8e2338cf4385a6fdd9cfc62b8e55dea6a7d69bf24ffc5aa6dfc18e1e06119ef5: Status 404 returned error can't find the container with id 8e2338cf4385a6fdd9cfc62b8e55dea6a7d69bf24ffc5aa6dfc18e1e06119ef5
Nov 03 22:59:11.767698 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.767639    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:59:11.767772 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.767732    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:59:11.767772 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.767762    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:59:11.767917 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.767888    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:59:11.767964 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.767952    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:59:11.767999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:11.767983    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 22:59:12.095216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.095123    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" event={"ID":"9d65809b-0adb-4d76-b9f5-ca690d30df9b","Type":"ContainerStarted","Data":"fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955"}
Nov 03 22:59:12.095216 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.095186    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" event={"ID":"9d65809b-0adb-4d76-b9f5-ca690d30df9b","Type":"ContainerStarted","Data":"8e2338cf4385a6fdd9cfc62b8e55dea6a7d69bf24ffc5aa6dfc18e1e06119ef5"}
Nov 03 22:59:12.097189 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.097160    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" event={"ID":"1a163038-fb12-44bf-9948-af32bee1357e","Type":"ContainerStarted","Data":"18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af"}
Nov 03 22:59:12.097333 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.097213    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" event={"ID":"1a163038-fb12-44bf-9948-af32bee1357e","Type":"ContainerStarted","Data":"d873f84d5608c3c3a248b148e268ef3d24c85aef353bf57a5cb221f8ea5cd622"}
Nov 03 22:59:12.220489 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.220428    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podStartSLOduration=2.220408573 podStartE2EDuration="2.220408573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:59:12.215178295 +0000 UTC m=+2175.655715558" watchObservedRunningTime="2026-11-03 22:59:12.220408573 +0000 UTC m=+2175.660945834"
Nov 03 22:59:12.267280 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.267244    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:12.269321 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.269286    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 03 22:59:12.269551 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.269354    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 03 22:59:12.316728 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.316548    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:12.322033 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.321994    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Nov 03 22:59:12.322247 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.322064    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Nov 03 22:59:12.340986 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:12.340914    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podStartSLOduration=2.340898171 podStartE2EDuration="2.340898171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:59:12.338010083 +0000 UTC m=+2175.778547344" watchObservedRunningTime="2026-11-03 22:59:12.340898171 +0000 UTC m=+2175.781435431"
Nov 03 22:59:13.268405 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:13.268376    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 03 22:59:13.268824 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:13.268420    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 03 22:59:13.316273 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:13.316234    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Nov 03 22:59:13.316579 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:13.316545    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Nov 03 22:59:14.268050 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:14.268016    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 03 22:59:14.268278 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:14.268076    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 03 22:59:14.320128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:14.320088    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Nov 03 22:59:14.320801 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:14.320768    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Nov 03 22:59:15.440701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:15.438768    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 03 22:59:15.440701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:15.438837    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 03 22:59:15.440701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:15.440281    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Nov 03 22:59:15.440701 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:15.440346    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Nov 03 22:59:15.592720 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:15.590871    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"]
Nov 03 22:59:15.664027 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:15.663994    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:16.046694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.046517    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ea5c8f-dockercfg-gzxdz\""
Nov 03 22:59:16.046694 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.046638    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ea5c8f\""
Nov 03 22:59:16.051740 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.048992    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f73fd7b6-a571-4586-bb8b-05b775f5b30c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw\" (UID: \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:16.051740 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.049081    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jf72\" (UniqueName: \"kubernetes.io/projected/f73fd7b6-a571-4586-bb8b-05b775f5b30c-kube-api-access-8jf72\") pod \"payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw\" (UID: \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:16.088528 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.086617    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"]
Nov 03 22:59:16.152729 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.152685    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f73fd7b6-a571-4586-bb8b-05b775f5b30c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw\" (UID: \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:16.156527 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.153867    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f73fd7b6-a571-4586-bb8b-05b775f5b30c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw\" (UID: \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:16.156797 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.156557    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jf72\" (UniqueName: \"kubernetes.io/projected/f73fd7b6-a571-4586-bb8b-05b775f5b30c-kube-api-access-8jf72\") pod \"payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw\" (UID: \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:16.228624 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.228581    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jf72\" (UniqueName: \"kubernetes.io/projected/f73fd7b6-a571-4586-bb8b-05b775f5b30c-kube-api-access-8jf72\") pod \"payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw\" (UID: \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:16.230365 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:59:16.230317    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:59:16.268898 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.268872    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 03 22:59:16.269048 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.268919    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 03 22:59:16.324859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.322888    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Nov 03 22:59:16.324859 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.322947    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Nov 03 22:59:16.458641 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.458611    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:16.662366 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.662333    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"]
Nov 03 22:59:16.662551 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:16.662526    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy" containerID="cri-o://243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a" gracePeriod=30
Nov 03 22:59:17.052993 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:17.052951    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"]
Nov 03 22:59:17.059092 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:59:17.059066    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73fd7b6_a571_4586_bb8b_05b775f5b30c.slice/crio-86f329eea6c8b38acd3b51e4a14dd5dea5af978bad0867cb731d29be397a0f0a WatchSource:0}: Error finding container 86f329eea6c8b38acd3b51e4a14dd5dea5af978bad0867cb731d29be397a0f0a: Status 404 returned error can't find the container with id 86f329eea6c8b38acd3b51e4a14dd5dea5af978bad0867cb731d29be397a0f0a
Nov 03 22:59:17.230970 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:17.230934    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw" event={"ID":"f73fd7b6-a571-4586-bb8b-05b775f5b30c","Type":"ContainerStarted","Data":"86f329eea6c8b38acd3b51e4a14dd5dea5af978bad0867cb731d29be397a0f0a"}
Nov 03 22:59:17.267625 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:17.267594    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 03 22:59:17.267822 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:17.267648    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 03 22:59:17.320241 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:17.320142    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Nov 03 22:59:17.320510 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:17.320484    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Nov 03 22:59:18.436470 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:18.307839    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Nov 03 22:59:18.436470 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:18.307899    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Nov 03 22:59:18.490697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:18.464072    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Nov 03 22:59:18.490697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:18.464130    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Nov 03 22:59:18.538259 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:18.538223    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw" event={"ID":"f73fd7b6-a571-4586-bb8b-05b775f5b30c","Type":"ContainerStarted","Data":"daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036"}
Nov 03 22:59:18.538572 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:18.538556    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:19.428040 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:19.428005    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:19.428652 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:19.428628    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:19.429691 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:19.429646    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 22:59:19.454532 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:19.454497    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:19.455008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:19.454990    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:19.456192 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:19.456168    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 22:59:19.548934 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:19.548886    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw" podStartSLOduration=4.548873775 podStartE2EDuration="4.548873775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:59:18.668835452 +0000 UTC m=+2182.109372711" watchObservedRunningTime="2026-11-03 22:59:19.548873775 +0000 UTC m=+2182.989411033"
Nov 03 22:59:21.951072 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:21.951013    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:59:22.099153 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099120    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-kube-api-access-rjh8n\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-kube-api-access-rjh8n\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099183    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-envoy\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099227    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-data\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099287    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istiod-ca-cert\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099337 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099321    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-token\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-token\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099421    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-certs\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099553 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099469    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-socket\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099689 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099591    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-podinfo\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099691    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-credential-socket\") pod \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\" (UID: \"a6d0312c-9d69-435c-a0b1-05ef3b52e454\") "
Nov 03 22:59:22.099757 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099729    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istiod-ca-cert" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 22:59:22.099923 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.099893    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-socket" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:59:22.100085 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.100035    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-certs" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:59:22.100308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.100257    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-credential-socket" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:59:22.100308 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.100267    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-data" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:59:22.100638 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.100617    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.100751 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.100644    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.100751 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.100660    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.100751 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.100714    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.100751 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.100727    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.102318 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.102293    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-kube-api-access-rjh8n" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "kube-api-access-rjh8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:59:22.102469 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.102370    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-podinfo" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 22:59:22.102610 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.102590    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-token" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 22:59:22.102723 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.102706    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-envoy" pod "a6d0312c-9d69-435c-a0b1-05ef3b52e454" (UID: "a6d0312c-9d69-435c-a0b1-05ef3b52e454"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 22:59:22.202684 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.202588    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjh8n\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-kube-api-access-rjh8n\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.202684 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.202618    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.202684 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.202628    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.202684 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.202637    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6d0312c-9d69-435c-a0b1-05ef3b52e454-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 22:59:22.570125 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.570092    5647 generic.go:356] "Generic (PLEG): container finished" podID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerID="243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a" exitCode=0
Nov 03 22:59:22.570310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.570138    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"
Nov 03 22:59:22.570310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.570174    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" event={"ID":"a6d0312c-9d69-435c-a0b1-05ef3b52e454","Type":"ContainerDied","Data":"243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a"}
Nov 03 22:59:22.570310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.570239    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb" event={"ID":"a6d0312c-9d69-435c-a0b1-05ef3b52e454","Type":"ContainerDied","Data":"25306d868f644938e438788ed27ec3550ceb0e2eaf1e8dc6a783e1f1c57e670b"}
Nov 03 22:59:22.570310 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.570269    5647 scope.go:122] "RemoveContainer" containerID="243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a"
Nov 03 22:59:22.582069 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.582052    5647 scope.go:122] "RemoveContainer" containerID="243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a"
Nov 03 22:59:22.582359 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:59:22.582341    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a\": container with ID starting with 243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a not found: ID does not exist" containerID="243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a"
Nov 03 22:59:22.582434 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.582368    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a"} err="failed to get container status \"243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a\": rpc error: code = NotFound desc = could not find container \"243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a\": container with ID starting with 243086d819b4b930f1bd7451a83b9a73cf1f373fe4ea827c13b3c959322d6a9a not found: ID does not exist"
Nov 03 22:59:22.738872 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.738842    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"]
Nov 03 22:59:22.788006 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:22.787955    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb"]
Nov 03 22:59:24.698128 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:24.698084    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" path="/var/lib/kubelet/pods/a6d0312c-9d69-435c-a0b1-05ef3b52e454/volumes"
Nov 03 22:59:27.772798 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:27.772758    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 22:59:29.572008 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:29.571966    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 22:59:34.651195 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.651162    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"]
Nov 03 22:59:34.651882 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.651867    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy"
Nov 03 22:59:34.651935 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.651883    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy"
Nov 03 22:59:34.652003 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.651994    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6d0312c-9d69-435c-a0b1-05ef3b52e454" containerName="istio-proxy"
Nov 03 22:59:34.657297 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.657276    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:34.716630 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.716593    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"]
Nov 03 22:59:34.760700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.760028    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1048fc\""
Nov 03 22:59:34.760700 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.760201    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1048fc-dockercfg-ql4vx\""
Nov 03 22:59:34.764699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.762112    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/987eea5f-e434-4274-99fe-b0213675ef77-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd\" (UID: \"987eea5f-e434-4274-99fe-b0213675ef77\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:34.764699 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.762305    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grfq7\" (UniqueName: \"kubernetes.io/projected/987eea5f-e434-4274-99fe-b0213675ef77-kube-api-access-grfq7\") pod \"payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd\" (UID: \"987eea5f-e434-4274-99fe-b0213675ef77\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:34.903789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.868605    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grfq7\" (UniqueName: \"kubernetes.io/projected/987eea5f-e434-4274-99fe-b0213675ef77-kube-api-access-grfq7\") pod \"payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd\" (UID: \"987eea5f-e434-4274-99fe-b0213675ef77\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:34.903789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.869003    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/987eea5f-e434-4274-99fe-b0213675ef77-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd\" (UID: \"987eea5f-e434-4274-99fe-b0213675ef77\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:34.903789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.870086    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/987eea5f-e434-4274-99fe-b0213675ef77-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd\" (UID: \"987eea5f-e434-4274-99fe-b0213675ef77\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:34.903789 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.887736    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"]
Nov 03 22:59:34.921936 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.921902    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"]
Nov 03 22:59:34.922126 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.922021    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:34.935456 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.935425    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grfq7\" (UniqueName: \"kubernetes.io/projected/987eea5f-e434-4274-99fe-b0213675ef77-kube-api-access-grfq7\") pod \"payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd\" (UID: \"987eea5f-e434-4274-99fe-b0213675ef77\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:34.971555 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.970181    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12830a1b-7618-41cf-860d-932a2ddd38a1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4\" (UID: \"12830a1b-7618-41cf-860d-932a2ddd38a1\") " pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:34.971555 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.970408    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqpxn\" (UniqueName: \"kubernetes.io/projected/12830a1b-7618-41cf-860d-932a2ddd38a1-kube-api-access-tqpxn\") pod \"payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4\" (UID: \"12830a1b-7618-41cf-860d-932a2ddd38a1\") " pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:34.971555 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:34.971466    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:35.077697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.076905    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqpxn\" (UniqueName: \"kubernetes.io/projected/12830a1b-7618-41cf-860d-932a2ddd38a1-kube-api-access-tqpxn\") pod \"payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4\" (UID: \"12830a1b-7618-41cf-860d-932a2ddd38a1\") " pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:35.077697 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.077039    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12830a1b-7618-41cf-860d-932a2ddd38a1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4\" (UID: \"12830a1b-7618-41cf-860d-932a2ddd38a1\") " pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:35.078727 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.078445    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12830a1b-7618-41cf-860d-932a2ddd38a1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4\" (UID: \"12830a1b-7618-41cf-860d-932a2ddd38a1\") " pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:35.139081 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.139050    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqpxn\" (UniqueName: \"kubernetes.io/projected/12830a1b-7618-41cf-860d-932a2ddd38a1-kube-api-access-tqpxn\") pod \"payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4\" (UID: \"12830a1b-7618-41cf-860d-932a2ddd38a1\") " pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:35.284404 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.284374    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:35.378782 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.378756    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"]
Nov 03 22:59:35.379178 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:59:35.379157    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987eea5f_e434_4274_99fe_b0213675ef77.slice/crio-79aef16e8538a9c2ee864dc930522f71c8d48703182f85acfb11ac4489bcf18a WatchSource:0}: Error finding container 79aef16e8538a9c2ee864dc930522f71c8d48703182f85acfb11ac4489bcf18a: Status 404 returned error can't find the container with id 79aef16e8538a9c2ee864dc930522f71c8d48703182f85acfb11ac4489bcf18a
Nov 03 22:59:35.666267 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.666237    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"]
Nov 03 22:59:35.666871 ip-10-0-132-187 kubenswrapper[5647]: W1103 22:59:35.666817    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12830a1b_7618_41cf_860d_932a2ddd38a1.slice/crio-a071cd1efb389efeac926e38973128e43b46d1c583de231d8ae125ed71fd3572 WatchSource:0}: Error finding container a071cd1efb389efeac926e38973128e43b46d1c583de231d8ae125ed71fd3572: Status 404 returned error can't find the container with id a071cd1efb389efeac926e38973128e43b46d1c583de231d8ae125ed71fd3572
Nov 03 22:59:35.677497 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.677466    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd" event={"ID":"987eea5f-e434-4274-99fe-b0213675ef77","Type":"ContainerStarted","Data":"39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a"}
Nov 03 22:59:35.677607 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.677527    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:35.677607 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.677549    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd" event={"ID":"987eea5f-e434-4274-99fe-b0213675ef77","Type":"ContainerStarted","Data":"79aef16e8538a9c2ee864dc930522f71c8d48703182f85acfb11ac4489bcf18a"}
Nov 03 22:59:35.794296 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:35.794248    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd" podStartSLOduration=1.794234262 podStartE2EDuration="1.794234262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:59:35.790809556 +0000 UTC m=+2199.231346816" watchObservedRunningTime="2026-11-03 22:59:35.794234262 +0000 UTC m=+2199.234771522"
Nov 03 22:59:36.685999 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:36.685962    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4" event={"ID":"12830a1b-7618-41cf-860d-932a2ddd38a1","Type":"ContainerStarted","Data":"a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc"}
Nov 03 22:59:36.686495 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:36.686050    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4" event={"ID":"12830a1b-7618-41cf-860d-932a2ddd38a1","Type":"ContainerStarted","Data":"a071cd1efb389efeac926e38973128e43b46d1c583de231d8ae125ed71fd3572"}
Nov 03 22:59:36.686495 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:36.686072    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 22:59:36.801175 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:36.801107    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4" podStartSLOduration=2.801089229 podStartE2EDuration="2.801089229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:59:36.800103123 +0000 UTC m=+2200.240640379" watchObservedRunningTime="2026-11-03 22:59:36.801089229 +0000 UTC m=+2200.241626489"
Nov 03 22:59:46.044368 ip-10-0-132-187 kubenswrapper[5647]: E1103 22:59:46.044327    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 22:59:46.698107 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:46.698080    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 22:59:47.696007 ip-10-0-132-187 kubenswrapper[5647]: I1103 22:59:47.695980    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 23:00:00.213868 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.213826    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"]
Nov 03 23:00:00.219904 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.219880    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"]
Nov 03 23:00:00.220057 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.219991    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:00.224314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.224294    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"]
Nov 03 23:00:00.224432 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.224417    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:00.228646 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.228626    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:00.264384 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.263209    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"]
Nov 03 23:00:00.268219 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.268197    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"]
Nov 03 23:00:00.268356 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.268322    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:00.272485 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.272464    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:00.296089 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.290899    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"]
Nov 03 23:00:00.296408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.296388    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:00.333145 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.332920    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"]
Nov 03 23:00:00.338364 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.338338    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"]
Nov 03 23:00:00.399410 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.398999    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d23630-c060-406d-b44e-68e52c9930fe-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw\" (UID: \"21d23630-c060-406d-b44e-68e52c9930fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:00.399410 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.399075    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dj9z\" (UniqueName: \"kubernetes.io/projected/c10abf5b-cf83-42b3-9f39-3814244a2422-kube-api-access-4dj9z\") pod \"maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh\" (UID: \"c10abf5b-cf83-42b3-9f39-3814244a2422\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:00.399410 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.399129    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb7d883-7387-46fe-b8ef-d753e9cee50b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m\" (UID: \"acb7d883-7387-46fe-b8ef-d753e9cee50b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:00.399410 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.399206    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5wb4\" (UniqueName: \"kubernetes.io/projected/17f8f446-d9af-4267-bda3-08a01ec2cfc1-kube-api-access-v5wb4\") pod \"maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg\" (UID: \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:00.400857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.399875    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pck9k\" (UniqueName: \"kubernetes.io/projected/acb7d883-7387-46fe-b8ef-d753e9cee50b-kube-api-access-pck9k\") pod \"maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m\" (UID: \"acb7d883-7387-46fe-b8ef-d753e9cee50b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:00.400857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.399923    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14400ebb-956e-45bc-94be-547e4aaf2cfa-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx\" (UID: \"14400ebb-956e-45bc-94be-547e4aaf2cfa\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:00.400857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.399969    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9r2q\" (UniqueName: \"kubernetes.io/projected/14400ebb-956e-45bc-94be-547e4aaf2cfa-kube-api-access-q9r2q\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx\" (UID: \"14400ebb-956e-45bc-94be-547e4aaf2cfa\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:00.400857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.400009    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10abf5b-cf83-42b3-9f39-3814244a2422-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh\" (UID: \"c10abf5b-cf83-42b3-9f39-3814244a2422\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:00.400857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.400060    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58vsx\" (UniqueName: \"kubernetes.io/projected/21d23630-c060-406d-b44e-68e52c9930fe-kube-api-access-58vsx\") pod \"maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw\" (UID: \"21d23630-c060-406d-b44e-68e52c9930fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:00.400857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.400136    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63cb0d01-a05c-490e-b069-8ce15ed85a24-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v\" (UID: \"63cb0d01-a05c-490e-b069-8ce15ed85a24\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:00.400857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.400337    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2sfh\" (UniqueName: \"kubernetes.io/projected/63cb0d01-a05c-490e-b069-8ce15ed85a24-kube-api-access-f2sfh\") pod \"maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v\" (UID: \"63cb0d01-a05c-490e-b069-8ce15ed85a24\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:00.400857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.400449    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17f8f446-d9af-4267-bda3-08a01ec2cfc1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg\" (UID: \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:00.401637 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.401482    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p2s2g\""
Nov 03 23:00:00.402585 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.402340    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"]
Nov 03 23:00:00.415716 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.414222    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"]
Nov 03 23:00:00.421485 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.421463    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"]
Nov 03 23:00:00.426022 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.426002    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"]
Nov 03 23:00:00.503302 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503226    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2sfh\" (UniqueName: \"kubernetes.io/projected/63cb0d01-a05c-490e-b069-8ce15ed85a24-kube-api-access-f2sfh\") pod \"maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v\" (UID: \"63cb0d01-a05c-490e-b069-8ce15ed85a24\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:00.503455 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503313    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17f8f446-d9af-4267-bda3-08a01ec2cfc1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg\" (UID: \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:00.503455 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503352    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d23630-c060-406d-b44e-68e52c9930fe-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw\" (UID: \"21d23630-c060-406d-b44e-68e52c9930fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:00.503455 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503384    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dj9z\" (UniqueName: \"kubernetes.io/projected/c10abf5b-cf83-42b3-9f39-3814244a2422-kube-api-access-4dj9z\") pod \"maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh\" (UID: \"c10abf5b-cf83-42b3-9f39-3814244a2422\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:00.503455 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503424    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb7d883-7387-46fe-b8ef-d753e9cee50b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m\" (UID: \"acb7d883-7387-46fe-b8ef-d753e9cee50b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:00.503643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503502    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5wb4\" (UniqueName: \"kubernetes.io/projected/17f8f446-d9af-4267-bda3-08a01ec2cfc1-kube-api-access-v5wb4\") pod \"maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg\" (UID: \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:00.503643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503530    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pck9k\" (UniqueName: \"kubernetes.io/projected/acb7d883-7387-46fe-b8ef-d753e9cee50b-kube-api-access-pck9k\") pod \"maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m\" (UID: \"acb7d883-7387-46fe-b8ef-d753e9cee50b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:00.503643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503551    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14400ebb-956e-45bc-94be-547e4aaf2cfa-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx\" (UID: \"14400ebb-956e-45bc-94be-547e4aaf2cfa\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:00.503928 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503584    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9r2q\" (UniqueName: \"kubernetes.io/projected/14400ebb-956e-45bc-94be-547e4aaf2cfa-kube-api-access-q9r2q\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx\" (UID: \"14400ebb-956e-45bc-94be-547e4aaf2cfa\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:00.504037 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.503971    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10abf5b-cf83-42b3-9f39-3814244a2422-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh\" (UID: \"c10abf5b-cf83-42b3-9f39-3814244a2422\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:00.504270 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.504246    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d23630-c060-406d-b44e-68e52c9930fe-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw\" (UID: \"21d23630-c060-406d-b44e-68e52c9930fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:00.504329 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.504246    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb7d883-7387-46fe-b8ef-d753e9cee50b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m\" (UID: \"acb7d883-7387-46fe-b8ef-d753e9cee50b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:00.504329 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.504295    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17f8f446-d9af-4267-bda3-08a01ec2cfc1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg\" (UID: \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:00.504418 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.504384    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58vsx\" (UniqueName: \"kubernetes.io/projected/21d23630-c060-406d-b44e-68e52c9930fe-kube-api-access-58vsx\") pod \"maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw\" (UID: \"21d23630-c060-406d-b44e-68e52c9930fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:00.504418 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.504396    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14400ebb-956e-45bc-94be-547e4aaf2cfa-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx\" (UID: \"14400ebb-956e-45bc-94be-547e4aaf2cfa\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:00.504518 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.504472    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63cb0d01-a05c-490e-b069-8ce15ed85a24-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v\" (UID: \"63cb0d01-a05c-490e-b069-8ce15ed85a24\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:00.504695 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.504644    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10abf5b-cf83-42b3-9f39-3814244a2422-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh\" (UID: \"c10abf5b-cf83-42b3-9f39-3814244a2422\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:00.505026 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.505009    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63cb0d01-a05c-490e-b069-8ce15ed85a24-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v\" (UID: \"63cb0d01-a05c-490e-b069-8ce15ed85a24\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:00.564605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.564562    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58vsx\" (UniqueName: \"kubernetes.io/projected/21d23630-c060-406d-b44e-68e52c9930fe-kube-api-access-58vsx\") pod \"maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw\" (UID: \"21d23630-c060-406d-b44e-68e52c9930fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:00.564605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.564586    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2sfh\" (UniqueName: \"kubernetes.io/projected/63cb0d01-a05c-490e-b069-8ce15ed85a24-kube-api-access-f2sfh\") pod \"maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v\" (UID: \"63cb0d01-a05c-490e-b069-8ce15ed85a24\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:00.564884 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.564699    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dj9z\" (UniqueName: \"kubernetes.io/projected/c10abf5b-cf83-42b3-9f39-3814244a2422-kube-api-access-4dj9z\") pod \"maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh\" (UID: \"c10abf5b-cf83-42b3-9f39-3814244a2422\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:00.564884 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.564807    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9r2q\" (UniqueName: \"kubernetes.io/projected/14400ebb-956e-45bc-94be-547e4aaf2cfa-kube-api-access-q9r2q\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx\" (UID: \"14400ebb-956e-45bc-94be-547e4aaf2cfa\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:00.564884 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.564826    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pck9k\" (UniqueName: \"kubernetes.io/projected/acb7d883-7387-46fe-b8ef-d753e9cee50b-kube-api-access-pck9k\") pod \"maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m\" (UID: \"acb7d883-7387-46fe-b8ef-d753e9cee50b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:00.565148 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.565132    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5wb4\" (UniqueName: \"kubernetes.io/projected/17f8f446-d9af-4267-bda3-08a01ec2cfc1-kube-api-access-v5wb4\") pod \"maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg\" (UID: \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:00.584148 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.584127    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:00.604305 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.604282    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:00.613847 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.613820    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:00.836547 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.836516    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:00.845063 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.845032    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:00.852835 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.852481    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:00.999863 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:00.971859    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"]
Nov 03 23:00:01.024137 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.021919    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"]
Nov 03 23:00:01.069617 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.069582    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"]
Nov 03 23:00:01.284680 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.284644    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"]
Nov 03 23:00:01.363358 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.363331    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"]
Nov 03 23:00:01.366455 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.366427    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"]
Nov 03 23:00:01.373866 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:00:01.373834    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10abf5b_cf83_42b3_9f39_3814244a2422.slice/crio-1c4b22d18b15d861f788b495ddb741c9c9522bd24d5633280699574eee09fb09 WatchSource:0}: Error finding container 1c4b22d18b15d861f788b495ddb741c9c9522bd24d5633280699574eee09fb09: Status 404 returned error can't find the container with id 1c4b22d18b15d861f788b495ddb741c9c9522bd24d5633280699574eee09fb09
Nov 03 23:00:01.374973 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:00:01.374945    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63cb0d01_a05c_490e_b069_8ce15ed85a24.slice/crio-46c54610705dc9ca878fc73ad0235c0179282c48659c8d6e0bb93c832fb1a7aa WatchSource:0}: Error finding container 46c54610705dc9ca878fc73ad0235c0179282c48659c8d6e0bb93c832fb1a7aa: Status 404 returned error can't find the container with id 46c54610705dc9ca878fc73ad0235c0179282c48659c8d6e0bb93c832fb1a7aa
Nov 03 23:00:01.869700 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.869577    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" event={"ID":"c10abf5b-cf83-42b3-9f39-3814244a2422","Type":"ContainerStarted","Data":"1c4b22d18b15d861f788b495ddb741c9c9522bd24d5633280699574eee09fb09"}
Nov 03 23:00:01.872230 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.871310    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" event={"ID":"17f8f446-d9af-4267-bda3-08a01ec2cfc1","Type":"ContainerStarted","Data":"b508f17b5900e6de0e4e6a92d66ac3ae641ced5ff9c76bf05fb0eec3ed213d92"}
Nov 03 23:00:01.872230 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.871363    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" event={"ID":"17f8f446-d9af-4267-bda3-08a01ec2cfc1","Type":"ContainerStarted","Data":"38f001778ed34e15048756a77d627151513b85d52b995decc655e107d69f6650"}
Nov 03 23:00:01.873998 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.873969    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" event={"ID":"acb7d883-7387-46fe-b8ef-d753e9cee50b","Type":"ContainerStarted","Data":"69a4e381e6f0aeed93384c3852d170c0f6ab5cf370e2f96c970056f72050970b"}
Nov 03 23:00:01.874147 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.874023    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" event={"ID":"acb7d883-7387-46fe-b8ef-d753e9cee50b","Type":"ContainerStarted","Data":"e1fc67b5ee98430f076389919cb92646fefd470e59e10373df6369a54a29a024"}
Nov 03 23:00:01.875869 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.875846    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" event={"ID":"21d23630-c060-406d-b44e-68e52c9930fe","Type":"ContainerStarted","Data":"991d50ab236cfb72bc594752dce27b9177af19386eede4337468f03fbd48d37f"}
Nov 03 23:00:01.876061 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.875998    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" event={"ID":"21d23630-c060-406d-b44e-68e52c9930fe","Type":"ContainerStarted","Data":"aaf9a8097c4d5cbe8a1e6e71643949285921caa8ee38ef1f21bc31b7af20a1ba"}
Nov 03 23:00:01.878064 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.878041    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" event={"ID":"14400ebb-956e-45bc-94be-547e4aaf2cfa","Type":"ContainerStarted","Data":"b64b7b036792079983ad7b5964c263174a840f51818c369aa0b5cec422e18224"}
Nov 03 23:00:01.878172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.878086    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" event={"ID":"14400ebb-956e-45bc-94be-547e4aaf2cfa","Type":"ContainerStarted","Data":"f9789fbcc9aaaa80e0f9e80961e9b3dae16914e002c4f0a40079c52af2f20fb6"}
Nov 03 23:00:01.879596 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.879575    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" event={"ID":"63cb0d01-a05c-490e-b069-8ce15ed85a24","Type":"ContainerStarted","Data":"91b506d01678bb75f4d9b6ca18a40567d4f3bac54799708b8018ccee6e7853a1"}
Nov 03 23:00:01.879740 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.879623    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" event={"ID":"63cb0d01-a05c-490e-b069-8ce15ed85a24","Type":"ContainerStarted","Data":"46c54610705dc9ca878fc73ad0235c0179282c48659c8d6e0bb93c832fb1a7aa"}
Nov 03 23:00:01.987027 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:01.986973    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" podStartSLOduration=1.986957623 podStartE2EDuration="1.986957623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:01.985535619 +0000 UTC m=+2225.426072882" watchObservedRunningTime="2026-11-03 23:00:01.986957623 +0000 UTC m=+2225.427494883"
Nov 03 23:00:02.101841 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.101790    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" podStartSLOduration=2.101776288 podStartE2EDuration="2.101776288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:02.098364451 +0000 UTC m=+2225.538901724" watchObservedRunningTime="2026-11-03 23:00:02.101776288 +0000 UTC m=+2225.542313547"
Nov 03 23:00:02.213817 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.213767    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" podStartSLOduration=2.2137513269999998 podStartE2EDuration="2.213751327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:02.211798131 +0000 UTC m=+2225.652335393" watchObservedRunningTime="2026-11-03 23:00:02.213751327 +0000 UTC m=+2225.654288585"
Nov 03 23:00:02.327075 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.327015    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" podStartSLOduration=2.326997924 podStartE2EDuration="2.326997924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:02.324571936 +0000 UTC m=+2225.765109195" watchObservedRunningTime="2026-11-03 23:00:02.326997924 +0000 UTC m=+2225.767535182"
Nov 03 23:00:02.440816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.440686    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" podStartSLOduration=2.440651778 podStartE2EDuration="2.440651778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:02.43886615 +0000 UTC m=+2225.879403433" watchObservedRunningTime="2026-11-03 23:00:02.440651778 +0000 UTC m=+2225.881189041"
Nov 03 23:00:02.887607 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.887570    5647 generic.go:356] "Generic (PLEG): container finished" podID="21d23630-c060-406d-b44e-68e52c9930fe" containerID="991d50ab236cfb72bc594752dce27b9177af19386eede4337468f03fbd48d37f" exitCode=60
Nov 03 23:00:02.887826 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.887663    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" event={"ID":"21d23630-c060-406d-b44e-68e52c9930fe","Type":"ContainerDied","Data":"991d50ab236cfb72bc594752dce27b9177af19386eede4337468f03fbd48d37f"}
Nov 03 23:00:02.887978 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.887959    5647 scope.go:122] "RemoveContainer" containerID="991d50ab236cfb72bc594752dce27b9177af19386eede4337468f03fbd48d37f"
Nov 03 23:00:02.889408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.889387    5647 generic.go:356] "Generic (PLEG): container finished" podID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerID="b64b7b036792079983ad7b5964c263174a840f51818c369aa0b5cec422e18224" exitCode=60
Nov 03 23:00:02.889528 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.889489    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" event={"ID":"14400ebb-956e-45bc-94be-547e4aaf2cfa","Type":"ContainerDied","Data":"b64b7b036792079983ad7b5964c263174a840f51818c369aa0b5cec422e18224"}
Nov 03 23:00:02.889762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.889744    5647 scope.go:122] "RemoveContainer" containerID="b64b7b036792079983ad7b5964c263174a840f51818c369aa0b5cec422e18224"
Nov 03 23:00:02.891163 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.891137    5647 generic.go:356] "Generic (PLEG): container finished" podID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerID="91b506d01678bb75f4d9b6ca18a40567d4f3bac54799708b8018ccee6e7853a1" exitCode=60
Nov 03 23:00:02.891275 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.891232    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" event={"ID":"63cb0d01-a05c-490e-b069-8ce15ed85a24","Type":"ContainerDied","Data":"91b506d01678bb75f4d9b6ca18a40567d4f3bac54799708b8018ccee6e7853a1"}
Nov 03 23:00:02.891468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.891450    5647 scope.go:122] "RemoveContainer" containerID="91b506d01678bb75f4d9b6ca18a40567d4f3bac54799708b8018ccee6e7853a1"
Nov 03 23:00:02.900298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.900153    5647 generic.go:356] "Generic (PLEG): container finished" podID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerID="78ce3a44c65355177f68e1bed7e124249512c4393be24bc9cf8c881a0e15b549" exitCode=60
Nov 03 23:00:02.900298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.900267    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" event={"ID":"c10abf5b-cf83-42b3-9f39-3814244a2422","Type":"ContainerDied","Data":"78ce3a44c65355177f68e1bed7e124249512c4393be24bc9cf8c881a0e15b549"}
Nov 03 23:00:02.904818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.900775    5647 scope.go:122] "RemoveContainer" containerID="78ce3a44c65355177f68e1bed7e124249512c4393be24bc9cf8c881a0e15b549"
Nov 03 23:00:02.909591 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.909566    5647 generic.go:356] "Generic (PLEG): container finished" podID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerID="b508f17b5900e6de0e4e6a92d66ac3ae641ced5ff9c76bf05fb0eec3ed213d92" exitCode=60
Nov 03 23:00:02.909709 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.909627    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" event={"ID":"17f8f446-d9af-4267-bda3-08a01ec2cfc1","Type":"ContainerDied","Data":"b508f17b5900e6de0e4e6a92d66ac3ae641ced5ff9c76bf05fb0eec3ed213d92"}
Nov 03 23:00:02.909893 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.909876    5647 scope.go:122] "RemoveContainer" containerID="b508f17b5900e6de0e4e6a92d66ac3ae641ced5ff9c76bf05fb0eec3ed213d92"
Nov 03 23:00:02.912960 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.911693    5647 generic.go:356] "Generic (PLEG): container finished" podID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerID="69a4e381e6f0aeed93384c3852d170c0f6ab5cf370e2f96c970056f72050970b" exitCode=60
Nov 03 23:00:02.912960 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.911740    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" event={"ID":"acb7d883-7387-46fe-b8ef-d753e9cee50b","Type":"ContainerDied","Data":"69a4e381e6f0aeed93384c3852d170c0f6ab5cf370e2f96c970056f72050970b"}
Nov 03 23:00:02.912960 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:02.912144    5647 scope.go:122] "RemoveContainer" containerID="69a4e381e6f0aeed93384c3852d170c0f6ab5cf370e2f96c970056f72050970b"
Nov 03 23:00:03.428790 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.428752    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"]
Nov 03 23:00:03.475837 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.474437    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:03.485618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.485580    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"]
Nov 03 23:00:03.537516 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.536526    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e66d59\""
Nov 03 23:00:03.567702 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.558941    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e66d59-dockercfg-bb65g\""
Nov 03 23:00:03.733795 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.733424    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5bpb\" (UniqueName: \"kubernetes.io/projected/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-kube-api-access-h5bpb\") pod \"payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7\" (UID: \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:03.733795 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.733579    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7\" (UID: \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:03.740481 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.740459    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"]
Nov 03 23:00:03.747137 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.746827    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"]
Nov 03 23:00:03.747137 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.746938    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:03.834986 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.834956    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsrm5\" (UniqueName: \"kubernetes.io/projected/3ca465ff-12ce-41df-8642-9b0cd07dc68b-kube-api-access-rsrm5\") pod \"payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp\" (UID: \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:03.835186 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.835026    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5bpb\" (UniqueName: \"kubernetes.io/projected/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-kube-api-access-h5bpb\") pod \"payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7\" (UID: \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:03.835186 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.835168    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ca465ff-12ce-41df-8642-9b0cd07dc68b-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp\" (UID: \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:03.835396 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.835373    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7\" (UID: \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:03.836065 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.836048    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7\" (UID: \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:03.895586 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.895558    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5bpb\" (UniqueName: \"kubernetes.io/projected/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-kube-api-access-h5bpb\") pod \"payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7\" (UID: \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:03.920862 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.920826    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" event={"ID":"c10abf5b-cf83-42b3-9f39-3814244a2422","Type":"ContainerStarted","Data":"f9035ca2126c270ce5eadbc470b73160b0c9d92603b650144ecb1e3570dabd60"}
Nov 03 23:00:03.922769 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.922741    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" event={"ID":"17f8f446-d9af-4267-bda3-08a01ec2cfc1","Type":"ContainerStarted","Data":"be78230663091df617fe68207590c0ee6d1a1010923ff05262f2611873ff7f51"}
Nov 03 23:00:03.924577 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.924553    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" event={"ID":"acb7d883-7387-46fe-b8ef-d753e9cee50b","Type":"ContainerStarted","Data":"22c031ce4ecff3dd35813d2fccd15e421689e16ec56caedc5ee829ab589df681"}
Nov 03 23:00:03.926408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.926385    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" event={"ID":"21d23630-c060-406d-b44e-68e52c9930fe","Type":"ContainerStarted","Data":"3f7712376f382e8c2facae87a8db049e2b49c3757b4c12282e40d9ac19da76e0"}
Nov 03 23:00:03.928287 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.928267    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" event={"ID":"14400ebb-956e-45bc-94be-547e4aaf2cfa","Type":"ContainerStarted","Data":"11ee77e238d3288fc5cbddf5c080919f3207e290c1d6d8faa7e75ef9ba39ad40"}
Nov 03 23:00:03.930228 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.930205    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" event={"ID":"63cb0d01-a05c-490e-b069-8ce15ed85a24","Type":"ContainerStarted","Data":"380a237c04155245bb50cf813dbc5603e4e99cae8772b429c8beaccac024de2f"}
Nov 03 23:00:03.936986 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.936964    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsrm5\" (UniqueName: \"kubernetes.io/projected/3ca465ff-12ce-41df-8642-9b0cd07dc68b-kube-api-access-rsrm5\") pod \"payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp\" (UID: \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:03.937641 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.937168    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ca465ff-12ce-41df-8642-9b0cd07dc68b-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp\" (UID: \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:03.938005 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.937987    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ca465ff-12ce-41df-8642-9b0cd07dc68b-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp\" (UID: \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:03.997809 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:03.997774    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsrm5\" (UniqueName: \"kubernetes.io/projected/3ca465ff-12ce-41df-8642-9b0cd07dc68b-kube-api-access-rsrm5\") pod \"payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp\" (UID: \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:04.065746 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.065567    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:04.108942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.108898    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:04.143511 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.143459    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" podStartSLOduration=4.143443323 podStartE2EDuration="4.143443323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:04.139826248 +0000 UTC m=+2227.580363512" watchObservedRunningTime="2026-11-03 23:00:04.143443323 +0000 UTC m=+2227.583980582"
Nov 03 23:00:04.459255 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.459230    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"]
Nov 03 23:00:04.459576 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:00:04.459398    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca465ff_12ce_41df_8642_9b0cd07dc68b.slice/crio-93fb9b6f4d1aa8a3bd95984f14241bced32204c2dc4f2fa1c7600f414b3e677e WatchSource:0}: Error finding container 93fb9b6f4d1aa8a3bd95984f14241bced32204c2dc4f2fa1c7600f414b3e677e: Status 404 returned error can't find the container with id 93fb9b6f4d1aa8a3bd95984f14241bced32204c2dc4f2fa1c7600f414b3e677e
Nov 03 23:00:04.519681 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.519647    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"]
Nov 03 23:00:04.521102 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:00:04.521080    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc98e8b3_e780_4bc2_bc7f_1db2f505becc.slice/crio-c82d7606293354263dd43d187134f9e5c7f76e26c39efedd9457c982974ba147 WatchSource:0}: Error finding container c82d7606293354263dd43d187134f9e5c7f76e26c39efedd9457c982974ba147: Status 404 returned error can't find the container with id c82d7606293354263dd43d187134f9e5c7f76e26c39efedd9457c982974ba147
Nov 03 23:00:04.938734 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.938626    5647 generic.go:356] "Generic (PLEG): container finished" podID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerID="f9035ca2126c270ce5eadbc470b73160b0c9d92603b650144ecb1e3570dabd60" exitCode=60
Nov 03 23:00:04.938734 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.938712    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" event={"ID":"c10abf5b-cf83-42b3-9f39-3814244a2422","Type":"ContainerDied","Data":"f9035ca2126c270ce5eadbc470b73160b0c9d92603b650144ecb1e3570dabd60"}
Nov 03 23:00:04.938956 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.938802    5647 scope.go:122] "RemoveContainer" containerID="78ce3a44c65355177f68e1bed7e124249512c4393be24bc9cf8c881a0e15b549"
Nov 03 23:00:04.939185 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.939154    5647 scope.go:122] "RemoveContainer" containerID="f9035ca2126c270ce5eadbc470b73160b0c9d92603b650144ecb1e3570dabd60"
Nov 03 23:00:04.939586 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:04.939562    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh_odh-ai-gateway-infra(c10abf5b-cf83-42b3-9f39-3814244a2422)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422"
Nov 03 23:00:04.940819 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.940795    5647 generic.go:356] "Generic (PLEG): container finished" podID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerID="be78230663091df617fe68207590c0ee6d1a1010923ff05262f2611873ff7f51" exitCode=60
Nov 03 23:00:04.940953 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.940867    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" event={"ID":"17f8f446-d9af-4267-bda3-08a01ec2cfc1","Type":"ContainerDied","Data":"be78230663091df617fe68207590c0ee6d1a1010923ff05262f2611873ff7f51"}
Nov 03 23:00:04.941147 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.941132    5647 scope.go:122] "RemoveContainer" containerID="be78230663091df617fe68207590c0ee6d1a1010923ff05262f2611873ff7f51"
Nov 03 23:00:04.941466 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:04.941448    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg_odh-ai-gateway-infra(17f8f446-d9af-4267-bda3-08a01ec2cfc1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1"
Nov 03 23:00:04.942691 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.942650    5647 generic.go:356] "Generic (PLEG): container finished" podID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerID="22c031ce4ecff3dd35813d2fccd15e421689e16ec56caedc5ee829ab589df681" exitCode=60
Nov 03 23:00:04.942803 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.942764    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" event={"ID":"acb7d883-7387-46fe-b8ef-d753e9cee50b","Type":"ContainerDied","Data":"22c031ce4ecff3dd35813d2fccd15e421689e16ec56caedc5ee829ab589df681"}
Nov 03 23:00:04.942988 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.942970    5647 scope.go:122] "RemoveContainer" containerID="22c031ce4ecff3dd35813d2fccd15e421689e16ec56caedc5ee829ab589df681"
Nov 03 23:00:04.943312 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:04.943288    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m_odh-ai-gateway-infra(acb7d883-7387-46fe-b8ef-d753e9cee50b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b"
Nov 03 23:00:04.944387 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.944365    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7" event={"ID":"bc98e8b3-e780-4bc2-bc7f-1db2f505becc","Type":"ContainerStarted","Data":"40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7"}
Nov 03 23:00:04.944488 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.944407    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7" event={"ID":"bc98e8b3-e780-4bc2-bc7f-1db2f505becc","Type":"ContainerStarted","Data":"c82d7606293354263dd43d187134f9e5c7f76e26c39efedd9457c982974ba147"}
Nov 03 23:00:04.944488 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.944436    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:04.946363 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.946341    5647 generic.go:356] "Generic (PLEG): container finished" podID="21d23630-c060-406d-b44e-68e52c9930fe" containerID="3f7712376f382e8c2facae87a8db049e2b49c3757b4c12282e40d9ac19da76e0" exitCode=60
Nov 03 23:00:04.946470 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.946447    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" event={"ID":"21d23630-c060-406d-b44e-68e52c9930fe","Type":"ContainerDied","Data":"3f7712376f382e8c2facae87a8db049e2b49c3757b4c12282e40d9ac19da76e0"}
Nov 03 23:00:04.946604 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.946591    5647 scope.go:122] "RemoveContainer" containerID="3f7712376f382e8c2facae87a8db049e2b49c3757b4c12282e40d9ac19da76e0"
Nov 03 23:00:04.946896 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:04.946869    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw_odh-ai-gateway-infra(21d23630-c060-406d-b44e-68e52c9930fe)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" podUID="21d23630-c060-406d-b44e-68e52c9930fe"
Nov 03 23:00:04.948360 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.948339    5647 generic.go:356] "Generic (PLEG): container finished" podID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerID="11ee77e238d3288fc5cbddf5c080919f3207e290c1d6d8faa7e75ef9ba39ad40" exitCode=60
Nov 03 23:00:04.948477 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.948413    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" event={"ID":"14400ebb-956e-45bc-94be-547e4aaf2cfa","Type":"ContainerDied","Data":"11ee77e238d3288fc5cbddf5c080919f3207e290c1d6d8faa7e75ef9ba39ad40"}
Nov 03 23:00:04.948630 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.948617    5647 scope.go:122] "RemoveContainer" containerID="11ee77e238d3288fc5cbddf5c080919f3207e290c1d6d8faa7e75ef9ba39ad40"
Nov 03 23:00:04.948924 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:04.948899    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx_odh-ai-gateway-infra(14400ebb-956e-45bc-94be-547e4aaf2cfa)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa"
Nov 03 23:00:04.950310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.950291    5647 generic.go:356] "Generic (PLEG): container finished" podID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerID="380a237c04155245bb50cf813dbc5603e4e99cae8772b429c8beaccac024de2f" exitCode=60
Nov 03 23:00:04.950401 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.950369    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" event={"ID":"63cb0d01-a05c-490e-b069-8ce15ed85a24","Type":"ContainerDied","Data":"380a237c04155245bb50cf813dbc5603e4e99cae8772b429c8beaccac024de2f"}
Nov 03 23:00:04.950532 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.950516    5647 scope.go:122] "RemoveContainer" containerID="380a237c04155245bb50cf813dbc5603e4e99cae8772b429c8beaccac024de2f"
Nov 03 23:00:04.950797 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:04.950776    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v_odh-ai-gateway-infra(63cb0d01-a05c-490e-b069-8ce15ed85a24)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24"
Nov 03 23:00:04.951945 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.951922    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp" event={"ID":"3ca465ff-12ce-41df-8642-9b0cd07dc68b","Type":"ContainerStarted","Data":"7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473"}
Nov 03 23:00:04.952033 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.951968    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp" event={"ID":"3ca465ff-12ce-41df-8642-9b0cd07dc68b","Type":"ContainerStarted","Data":"93fb9b6f4d1aa8a3bd95984f14241bced32204c2dc4f2fa1c7600f414b3e677e"}
Nov 03 23:00:04.952033 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.951996    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:04.954059 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.954033    5647 scope.go:122] "RemoveContainer" containerID="b508f17b5900e6de0e4e6a92d66ac3ae641ced5ff9c76bf05fb0eec3ed213d92"
Nov 03 23:00:04.970937 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.970914    5647 scope.go:122] "RemoveContainer" containerID="69a4e381e6f0aeed93384c3852d170c0f6ab5cf370e2f96c970056f72050970b"
Nov 03 23:00:04.985886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.985863    5647 scope.go:122] "RemoveContainer" containerID="991d50ab236cfb72bc594752dce27b9177af19386eede4337468f03fbd48d37f"
Nov 03 23:00:04.999642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:04.999622    5647 scope.go:122] "RemoveContainer" containerID="b64b7b036792079983ad7b5964c263174a840f51818c369aa0b5cec422e18224"
Nov 03 23:00:05.020226 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.020205    5647 scope.go:122] "RemoveContainer" containerID="91b506d01678bb75f4d9b6ca18a40567d4f3bac54799708b8018ccee6e7853a1"
Nov 03 23:00:05.288461 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.288407    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7" podStartSLOduration=2.288393988 podStartE2EDuration="2.288393988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:05.28456384 +0000 UTC m=+2228.725101100" watchObservedRunningTime="2026-11-03 23:00:05.288393988 +0000 UTC m=+2228.728931248"
Nov 03 23:00:05.868781 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.868723    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp" podStartSLOduration=2.868700746 podStartE2EDuration="2.868700746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:05.866305808 +0000 UTC m=+2229.306843080" watchObservedRunningTime="2026-11-03 23:00:05.868700746 +0000 UTC m=+2229.309238027"
Nov 03 23:00:05.962529 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.962487    5647 scope.go:122] "RemoveContainer" containerID="be78230663091df617fe68207590c0ee6d1a1010923ff05262f2611873ff7f51"
Nov 03 23:00:05.962907 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:05.962884    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg_odh-ai-gateway-infra(17f8f446-d9af-4267-bda3-08a01ec2cfc1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1"
Nov 03 23:00:05.964157 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.964134    5647 scope.go:122] "RemoveContainer" containerID="22c031ce4ecff3dd35813d2fccd15e421689e16ec56caedc5ee829ab589df681"
Nov 03 23:00:05.964483 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:05.964461    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m_odh-ai-gateway-infra(acb7d883-7387-46fe-b8ef-d753e9cee50b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b"
Nov 03 23:00:05.965681 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.965647    5647 scope.go:122] "RemoveContainer" containerID="3f7712376f382e8c2facae87a8db049e2b49c3757b4c12282e40d9ac19da76e0"
Nov 03 23:00:05.965997 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:05.965970    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw_odh-ai-gateway-infra(21d23630-c060-406d-b44e-68e52c9930fe)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" podUID="21d23630-c060-406d-b44e-68e52c9930fe"
Nov 03 23:00:05.967137 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.967121    5647 scope.go:122] "RemoveContainer" containerID="11ee77e238d3288fc5cbddf5c080919f3207e290c1d6d8faa7e75ef9ba39ad40"
Nov 03 23:00:05.967431 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:05.967404    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx_odh-ai-gateway-infra(14400ebb-956e-45bc-94be-547e4aaf2cfa)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa"
Nov 03 23:00:05.968849 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.968829    5647 scope.go:122] "RemoveContainer" containerID="380a237c04155245bb50cf813dbc5603e4e99cae8772b429c8beaccac024de2f"
Nov 03 23:00:05.969187 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:05.969165    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v_odh-ai-gateway-infra(63cb0d01-a05c-490e-b069-8ce15ed85a24)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24"
Nov 03 23:00:05.970550 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:05.970533    5647 scope.go:122] "RemoveContainer" containerID="f9035ca2126c270ce5eadbc470b73160b0c9d92603b650144ecb1e3570dabd60"
Nov 03 23:00:05.970748 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:05.970730    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh_odh-ai-gateway-infra(c10abf5b-cf83-42b3-9f39-3814244a2422)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422"
Nov 03 23:00:15.697758 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:15.689124    5647 scope.go:122] "RemoveContainer" containerID="11ee77e238d3288fc5cbddf5c080919f3207e290c1d6d8faa7e75ef9ba39ad40"
Nov 03 23:00:15.697758 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:15.690159    5647 scope.go:122] "RemoveContainer" containerID="f9035ca2126c270ce5eadbc470b73160b0c9d92603b650144ecb1e3570dabd60"
Nov 03 23:00:15.697758 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:15.690932    5647 scope.go:122] "RemoveContainer" containerID="22c031ce4ecff3dd35813d2fccd15e421689e16ec56caedc5ee829ab589df681"
Nov 03 23:00:16.005947 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:16.005790    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:00:16.021704 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:16.018983    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:00:16.154688 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:16.154634    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:00:16.697750 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:16.693286    5647 scope.go:122] "RemoveContainer" containerID="3f7712376f382e8c2facae87a8db049e2b49c3757b4c12282e40d9ac19da76e0"
Nov 03 23:00:16.702504 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:16.702484    5647 scope.go:122] "RemoveContainer" containerID="be78230663091df617fe68207590c0ee6d1a1010923ff05262f2611873ff7f51"
Nov 03 23:00:17.197149 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.197073    5647 generic.go:356] "Generic (PLEG): container finished" podID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerID="3e098cb706b2a3d4e08a2770c1c30dc8e245786d4d966d58204c5b7d8ce31b4a" exitCode=60
Nov 03 23:00:17.197392 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.197176    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" event={"ID":"c10abf5b-cf83-42b3-9f39-3814244a2422","Type":"ContainerDied","Data":"3e098cb706b2a3d4e08a2770c1c30dc8e245786d4d966d58204c5b7d8ce31b4a"}
Nov 03 23:00:17.197392 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.197240    5647 scope.go:122] "RemoveContainer" containerID="f9035ca2126c270ce5eadbc470b73160b0c9d92603b650144ecb1e3570dabd60"
Nov 03 23:00:17.197865 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.197834    5647 scope.go:122] "RemoveContainer" containerID="3e098cb706b2a3d4e08a2770c1c30dc8e245786d4d966d58204c5b7d8ce31b4a"
Nov 03 23:00:17.198582 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:17.198261    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh_odh-ai-gateway-infra(c10abf5b-cf83-42b3-9f39-3814244a2422)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422"
Nov 03 23:00:17.206250 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.206227    5647 generic.go:356] "Generic (PLEG): container finished" podID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerID="ec361faf8d8549a6a0fe4046a1f65ad6620fc49af59fae7cb2804ced92f721d1" exitCode=60
Nov 03 23:00:17.206383 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.206325    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" event={"ID":"acb7d883-7387-46fe-b8ef-d753e9cee50b","Type":"ContainerDied","Data":"ec361faf8d8549a6a0fe4046a1f65ad6620fc49af59fae7cb2804ced92f721d1"}
Nov 03 23:00:17.206755 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.206738    5647 scope.go:122] "RemoveContainer" containerID="ec361faf8d8549a6a0fe4046a1f65ad6620fc49af59fae7cb2804ced92f721d1"
Nov 03 23:00:17.207129 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:17.207110    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m_odh-ai-gateway-infra(acb7d883-7387-46fe-b8ef-d753e9cee50b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b"
Nov 03 23:00:17.210092 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.210067    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" event={"ID":"21d23630-c060-406d-b44e-68e52c9930fe","Type":"ContainerStarted","Data":"84ff1753710a647d44d2e7d5f6f46d2fdc1a4abf3d6dbed4636fe32870104dfa"}
Nov 03 23:00:17.213130 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.213108    5647 generic.go:356] "Generic (PLEG): container finished" podID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerID="8e79e06bf2d283731ef0e6fa4b962b6dfe2fb62453ada7b092d0aa682e060020" exitCode=60
Nov 03 23:00:17.213232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.213193    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" event={"ID":"14400ebb-956e-45bc-94be-547e4aaf2cfa","Type":"ContainerDied","Data":"8e79e06bf2d283731ef0e6fa4b962b6dfe2fb62453ada7b092d0aa682e060020"}
Nov 03 23:00:17.213499 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.213485    5647 scope.go:122] "RemoveContainer" containerID="8e79e06bf2d283731ef0e6fa4b962b6dfe2fb62453ada7b092d0aa682e060020"
Nov 03 23:00:17.213774 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:17.213755    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx_odh-ai-gateway-infra(14400ebb-956e-45bc-94be-547e4aaf2cfa)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa"
Nov 03 23:00:17.219050 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.219028    5647 scope.go:122] "RemoveContainer" containerID="22c031ce4ecff3dd35813d2fccd15e421689e16ec56caedc5ee829ab589df681"
Nov 03 23:00:17.232640 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:17.232621    5647 scope.go:122] "RemoveContainer" containerID="11ee77e238d3288fc5cbddf5c080919f3207e290c1d6d8faa7e75ef9ba39ad40"
Nov 03 23:00:18.228232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.228198    5647 generic.go:356] "Generic (PLEG): container finished" podID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerID="2ac8f98dda1e48b9c0662473b0d776f1542e2e29f6f7d354098bbd3851ff1212" exitCode=60
Nov 03 23:00:18.228723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.228263    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" event={"ID":"17f8f446-d9af-4267-bda3-08a01ec2cfc1","Type":"ContainerDied","Data":"2ac8f98dda1e48b9c0662473b0d776f1542e2e29f6f7d354098bbd3851ff1212"}
Nov 03 23:00:18.228723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.228353    5647 scope.go:122] "RemoveContainer" containerID="be78230663091df617fe68207590c0ee6d1a1010923ff05262f2611873ff7f51"
Nov 03 23:00:18.228898 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.228882    5647 scope.go:122] "RemoveContainer" containerID="2ac8f98dda1e48b9c0662473b0d776f1542e2e29f6f7d354098bbd3851ff1212"
Nov 03 23:00:18.229719 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:18.229328    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg_odh-ai-gateway-infra(17f8f446-d9af-4267-bda3-08a01ec2cfc1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1"
Nov 03 23:00:18.231593 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.231571    5647 generic.go:356] "Generic (PLEG): container finished" podID="21d23630-c060-406d-b44e-68e52c9930fe" containerID="84ff1753710a647d44d2e7d5f6f46d2fdc1a4abf3d6dbed4636fe32870104dfa" exitCode=60
Nov 03 23:00:18.231735 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.231616    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" event={"ID":"21d23630-c060-406d-b44e-68e52c9930fe","Type":"ContainerDied","Data":"84ff1753710a647d44d2e7d5f6f46d2fdc1a4abf3d6dbed4636fe32870104dfa"}
Nov 03 23:00:18.232009 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.231992    5647 scope.go:122] "RemoveContainer" containerID="84ff1753710a647d44d2e7d5f6f46d2fdc1a4abf3d6dbed4636fe32870104dfa"
Nov 03 23:00:18.232297 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:18.232278    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw_odh-ai-gateway-infra(21d23630-c060-406d-b44e-68e52c9930fe)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" podUID="21d23630-c060-406d-b44e-68e52c9930fe"
Nov 03 23:00:18.242426 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.242405    5647 scope.go:122] "RemoveContainer" containerID="3f7712376f382e8c2facae87a8db049e2b49c3757b4c12282e40d9ac19da76e0"
Nov 03 23:00:18.402053 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.401972    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"]
Nov 03 23:00:18.530078 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.530044    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"]
Nov 03 23:00:18.634216 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.634185    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"]
Nov 03 23:00:18.704617 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.703181    5647 scope.go:122] "RemoveContainer" containerID="380a237c04155245bb50cf813dbc5603e4e99cae8772b429c8beaccac024de2f"
Nov 03 23:00:18.714571 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.713032    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:18.763608 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.763577    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"]
Nov 03 23:00:18.822089 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.822062    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:18.886693 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.870452    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c10abf5b-cf83-42b3-9f39-3814244a2422-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10abf5b-cf83-42b3-9f39-3814244a2422-maas-api-service-ca\") pod \"c10abf5b-cf83-42b3-9f39-3814244a2422\" (UID: \"c10abf5b-cf83-42b3-9f39-3814244a2422\") "
Nov 03 23:00:18.886693 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.870621    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c10abf5b-cf83-42b3-9f39-3814244a2422-kube-api-access-4dj9z\" (UniqueName: \"kubernetes.io/projected/c10abf5b-cf83-42b3-9f39-3814244a2422-kube-api-access-4dj9z\") pod \"c10abf5b-cf83-42b3-9f39-3814244a2422\" (UID: \"c10abf5b-cf83-42b3-9f39-3814244a2422\") "
Nov 03 23:00:18.886693 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.877875    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c10abf5b-cf83-42b3-9f39-3814244a2422-maas-api-service-ca" pod "c10abf5b-cf83-42b3-9f39-3814244a2422" (UID: "c10abf5b-cf83-42b3-9f39-3814244a2422"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:18.904890 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.904631    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c10abf5b-cf83-42b3-9f39-3814244a2422-kube-api-access-4dj9z" pod "c10abf5b-cf83-42b3-9f39-3814244a2422" (UID: "c10abf5b-cf83-42b3-9f39-3814244a2422"). InnerVolumeSpecName "kube-api-access-4dj9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:18.951257 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.951228    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:18.995749 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.990231    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14400ebb-956e-45bc-94be-547e4aaf2cfa-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14400ebb-956e-45bc-94be-547e4aaf2cfa-maas-api-service-ca\") pod \"14400ebb-956e-45bc-94be-547e4aaf2cfa\" (UID: \"14400ebb-956e-45bc-94be-547e4aaf2cfa\") "
Nov 03 23:00:18.995749 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.990295    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14400ebb-956e-45bc-94be-547e4aaf2cfa-kube-api-access-q9r2q\" (UniqueName: \"kubernetes.io/projected/14400ebb-956e-45bc-94be-547e4aaf2cfa-kube-api-access-q9r2q\") pod \"14400ebb-956e-45bc-94be-547e4aaf2cfa\" (UID: \"14400ebb-956e-45bc-94be-547e4aaf2cfa\") "
Nov 03 23:00:18.995749 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.991316    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10abf5b-cf83-42b3-9f39-3814244a2422-maas-api-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:18.995749 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.991343    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dj9z\" (UniqueName: \"kubernetes.io/projected/c10abf5b-cf83-42b3-9f39-3814244a2422-kube-api-access-4dj9z\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:18.996089 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.996006    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14400ebb-956e-45bc-94be-547e4aaf2cfa-maas-api-service-ca" pod "14400ebb-956e-45bc-94be-547e4aaf2cfa" (UID: "14400ebb-956e-45bc-94be-547e4aaf2cfa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:18.999517 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:18.999487    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14400ebb-956e-45bc-94be-547e4aaf2cfa-kube-api-access-q9r2q" pod "14400ebb-956e-45bc-94be-547e4aaf2cfa" (UID: "14400ebb-956e-45bc-94be-547e4aaf2cfa"). InnerVolumeSpecName "kube-api-access-q9r2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:19.098712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.092288    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acb7d883-7387-46fe-b8ef-d753e9cee50b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb7d883-7387-46fe-b8ef-d753e9cee50b-maas-api-service-ca\") pod \"acb7d883-7387-46fe-b8ef-d753e9cee50b\" (UID: \"acb7d883-7387-46fe-b8ef-d753e9cee50b\") "
Nov 03 23:00:19.098712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.092477    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acb7d883-7387-46fe-b8ef-d753e9cee50b-kube-api-access-pck9k\" (UniqueName: \"kubernetes.io/projected/acb7d883-7387-46fe-b8ef-d753e9cee50b-kube-api-access-pck9k\") pod \"acb7d883-7387-46fe-b8ef-d753e9cee50b\" (UID: \"acb7d883-7387-46fe-b8ef-d753e9cee50b\") "
Nov 03 23:00:19.098712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.093327    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14400ebb-956e-45bc-94be-547e4aaf2cfa-maas-api-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.098712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.093353    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9r2q\" (UniqueName: \"kubernetes.io/projected/14400ebb-956e-45bc-94be-547e4aaf2cfa-kube-api-access-q9r2q\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.098712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.097825    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acb7d883-7387-46fe-b8ef-d753e9cee50b-maas-api-service-ca" pod "acb7d883-7387-46fe-b8ef-d753e9cee50b" (UID: "acb7d883-7387-46fe-b8ef-d753e9cee50b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:19.099291 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.099256    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acb7d883-7387-46fe-b8ef-d753e9cee50b-kube-api-access-pck9k" pod "acb7d883-7387-46fe-b8ef-d753e9cee50b" (UID: "acb7d883-7387-46fe-b8ef-d753e9cee50b"). InnerVolumeSpecName "kube-api-access-pck9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:19.199762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.197291    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pck9k\" (UniqueName: \"kubernetes.io/projected/acb7d883-7387-46fe-b8ef-d753e9cee50b-kube-api-access-pck9k\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.199958 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.199764    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb7d883-7387-46fe-b8ef-d753e9cee50b-maas-api-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.250260 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.250190    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"
Nov 03 23:00:19.250806 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.250773    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx" event={"ID":"14400ebb-956e-45bc-94be-547e4aaf2cfa","Type":"ContainerDied","Data":"f9789fbcc9aaaa80e0f9e80961e9b3dae16914e002c4f0a40079c52af2f20fb6"}
Nov 03 23:00:19.250879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.250833    5647 scope.go:122] "RemoveContainer" containerID="8e79e06bf2d283731ef0e6fa4b962b6dfe2fb62453ada7b092d0aa682e060020"
Nov 03 23:00:19.255911 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.255881    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" event={"ID":"63cb0d01-a05c-490e-b069-8ce15ed85a24","Type":"ContainerStarted","Data":"8bf648cd7d5a0489af5df3ad9aec259ed7b8db7209992f50f242aca6e1b246f4"}
Nov 03 23:00:19.258239 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.258208    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh" event={"ID":"c10abf5b-cf83-42b3-9f39-3814244a2422","Type":"ContainerDied","Data":"1c4b22d18b15d861f788b495ddb741c9c9522bd24d5633280699574eee09fb09"}
Nov 03 23:00:19.258346 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.258308    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"
Nov 03 23:00:19.262210 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.262188    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m" event={"ID":"acb7d883-7387-46fe-b8ef-d753e9cee50b","Type":"ContainerDied","Data":"e1fc67b5ee98430f076389919cb92646fefd470e59e10373df6369a54a29a024"}
Nov 03 23:00:19.262305 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.262198    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"
Nov 03 23:00:19.262661 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.262640    5647 scope.go:122] "RemoveContainer" containerID="3e098cb706b2a3d4e08a2770c1c30dc8e245786d4d966d58204c5b7d8ce31b4a"
Nov 03 23:00:19.310924 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.308483    5647 scope.go:122] "RemoveContainer" containerID="ec361faf8d8549a6a0fe4046a1f65ad6620fc49af59fae7cb2804ced92f721d1"
Nov 03 23:00:19.359533 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.359511    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"]
Nov 03 23:00:19.416586 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.416551    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"]
Nov 03 23:00:19.420943 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.420917    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-433db8-29895780-j6fcx"]
Nov 03 23:00:19.526942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.526869    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"]
Nov 03 23:00:19.529125 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.529037    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:19.535554 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.535530    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-164c36-29895780-6rllh"]
Nov 03 23:00:19.611028 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.611006    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:19.717233 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.717201    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17f8f446-d9af-4267-bda3-08a01ec2cfc1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17f8f446-d9af-4267-bda3-08a01ec2cfc1-maas-api-service-ca\") pod \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\" (UID: \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\") "
Nov 03 23:00:19.717422 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.717273    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21d23630-c060-406d-b44e-68e52c9930fe-kube-api-access-58vsx\" (UniqueName: \"kubernetes.io/projected/21d23630-c060-406d-b44e-68e52c9930fe-kube-api-access-58vsx\") pod \"21d23630-c060-406d-b44e-68e52c9930fe\" (UID: \"21d23630-c060-406d-b44e-68e52c9930fe\") "
Nov 03 23:00:19.717422 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.717348    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21d23630-c060-406d-b44e-68e52c9930fe-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d23630-c060-406d-b44e-68e52c9930fe-maas-api-service-ca\") pod \"21d23630-c060-406d-b44e-68e52c9930fe\" (UID: \"21d23630-c060-406d-b44e-68e52c9930fe\") "
Nov 03 23:00:19.717508 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.717495    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17f8f446-d9af-4267-bda3-08a01ec2cfc1-kube-api-access-v5wb4\" (UniqueName: \"kubernetes.io/projected/17f8f446-d9af-4267-bda3-08a01ec2cfc1-kube-api-access-v5wb4\") pod \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\" (UID: \"17f8f446-d9af-4267-bda3-08a01ec2cfc1\") "
Nov 03 23:00:19.717645 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.717625    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17f8f446-d9af-4267-bda3-08a01ec2cfc1-maas-api-service-ca" pod "17f8f446-d9af-4267-bda3-08a01ec2cfc1" (UID: "17f8f446-d9af-4267-bda3-08a01ec2cfc1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:19.717785 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.717729    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d23630-c060-406d-b44e-68e52c9930fe-maas-api-service-ca" pod "21d23630-c060-406d-b44e-68e52c9930fe" (UID: "21d23630-c060-406d-b44e-68e52c9930fe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:19.718179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.718155    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17f8f446-d9af-4267-bda3-08a01ec2cfc1-maas-api-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.718179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.718173    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d23630-c060-406d-b44e-68e52c9930fe-maas-api-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.719737 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.719712    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d23630-c060-406d-b44e-68e52c9930fe-kube-api-access-58vsx" pod "21d23630-c060-406d-b44e-68e52c9930fe" (UID: "21d23630-c060-406d-b44e-68e52c9930fe"). InnerVolumeSpecName "kube-api-access-58vsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:19.719820 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.719767    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f8f446-d9af-4267-bda3-08a01ec2cfc1-kube-api-access-v5wb4" pod "17f8f446-d9af-4267-bda3-08a01ec2cfc1" (UID: "17f8f446-d9af-4267-bda3-08a01ec2cfc1"). InnerVolumeSpecName "kube-api-access-v5wb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:19.819881 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.819792    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5wb4\" (UniqueName: \"kubernetes.io/projected/17f8f446-d9af-4267-bda3-08a01ec2cfc1-kube-api-access-v5wb4\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:19.819881 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:19.819820    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58vsx\" (UniqueName: \"kubernetes.io/projected/21d23630-c060-406d-b44e-68e52c9930fe-kube-api-access-58vsx\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:20.106191 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.106100    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"]
Nov 03 23:00:20.111815 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.111777    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-32449d-29895780-2t95m"]
Nov 03 23:00:20.300558 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.300524    5647 generic.go:356] "Generic (PLEG): container finished" podID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerID="8bf648cd7d5a0489af5df3ad9aec259ed7b8db7209992f50f242aca6e1b246f4" exitCode=60
Nov 03 23:00:20.301083 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.300819    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" event={"ID":"63cb0d01-a05c-490e-b069-8ce15ed85a24","Type":"ContainerDied","Data":"8bf648cd7d5a0489af5df3ad9aec259ed7b8db7209992f50f242aca6e1b246f4"}
Nov 03 23:00:20.301083 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.300892    5647 scope.go:122] "RemoveContainer" containerID="380a237c04155245bb50cf813dbc5603e4e99cae8772b429c8beaccac024de2f"
Nov 03 23:00:20.301360 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.301342    5647 scope.go:122] "RemoveContainer" containerID="8bf648cd7d5a0489af5df3ad9aec259ed7b8db7209992f50f242aca6e1b246f4"
Nov 03 23:00:20.301808 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:20.301780    5647 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v_odh-ai-gateway-infra(63cb0d01-a05c-490e-b069-8ce15ed85a24)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24"
Nov 03 23:00:20.303818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.303787    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg" event={"ID":"17f8f446-d9af-4267-bda3-08a01ec2cfc1","Type":"ContainerDied","Data":"38f001778ed34e15048756a77d627151513b85d52b995decc655e107d69f6650"}
Nov 03 23:00:20.303935 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.303789    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"
Nov 03 23:00:20.306379 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.306361    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"
Nov 03 23:00:20.306505 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.306404    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw" event={"ID":"21d23630-c060-406d-b44e-68e52c9930fe","Type":"ContainerDied","Data":"aaf9a8097c4d5cbe8a1e6e71643949285921caa8ee38ef1f21bc31b7af20a1ba"}
Nov 03 23:00:20.316281 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.316261    5647 scope.go:122] "RemoveContainer" containerID="2ac8f98dda1e48b9c0662473b0d776f1542e2e29f6f7d354098bbd3851ff1212"
Nov 03 23:00:20.327319 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.327297    5647 scope.go:122] "RemoveContainer" containerID="84ff1753710a647d44d2e7d5f6f46d2fdc1a4abf3d6dbed4636fe32870104dfa"
Nov 03 23:00:20.531726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.531570    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"]
Nov 03 23:00:20.565087 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.564976    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-f8cd0c-29895780-ldpbw"]
Nov 03 23:00:20.662535 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.661373    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"]
Nov 03 23:00:20.664919 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.664889    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-b64e2e-29895780-rggvg"]
Nov 03 23:00:20.705916 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.705878    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" path="/var/lib/kubelet/pods/14400ebb-956e-45bc-94be-547e4aaf2cfa/volumes"
Nov 03 23:00:20.706845 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.706821    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" path="/var/lib/kubelet/pods/17f8f446-d9af-4267-bda3-08a01ec2cfc1/volumes"
Nov 03 23:00:20.707729 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.707709    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21d23630-c060-406d-b44e-68e52c9930fe" path="/var/lib/kubelet/pods/21d23630-c060-406d-b44e-68e52c9930fe/volumes"
Nov 03 23:00:20.708777 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.708761    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" path="/var/lib/kubelet/pods/acb7d883-7387-46fe-b8ef-d753e9cee50b/volumes"
Nov 03 23:00:20.709265 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.709250    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" path="/var/lib/kubelet/pods/c10abf5b-cf83-42b3-9f39-3814244a2422/volumes"
Nov 03 23:00:20.733624 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:20.733595    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"]
Nov 03 23:00:21.510108 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:21.510076    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:21.604527 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:21.604495    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63cb0d01-a05c-490e-b069-8ce15ed85a24-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63cb0d01-a05c-490e-b069-8ce15ed85a24-maas-api-service-ca\") pod \"63cb0d01-a05c-490e-b069-8ce15ed85a24\" (UID: \"63cb0d01-a05c-490e-b069-8ce15ed85a24\") "
Nov 03 23:00:21.604735 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:21.604610    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63cb0d01-a05c-490e-b069-8ce15ed85a24-kube-api-access-f2sfh\" (UniqueName: \"kubernetes.io/projected/63cb0d01-a05c-490e-b069-8ce15ed85a24-kube-api-access-f2sfh\") pod \"63cb0d01-a05c-490e-b069-8ce15ed85a24\" (UID: \"63cb0d01-a05c-490e-b069-8ce15ed85a24\") "
Nov 03 23:00:21.604973 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:21.604946    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63cb0d01-a05c-490e-b069-8ce15ed85a24-maas-api-service-ca" pod "63cb0d01-a05c-490e-b069-8ce15ed85a24" (UID: "63cb0d01-a05c-490e-b069-8ce15ed85a24"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:21.605335 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:21.605311    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63cb0d01-a05c-490e-b069-8ce15ed85a24-maas-api-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:21.607562 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:21.607533    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63cb0d01-a05c-490e-b069-8ce15ed85a24-kube-api-access-f2sfh" pod "63cb0d01-a05c-490e-b069-8ce15ed85a24" (UID: "63cb0d01-a05c-490e-b069-8ce15ed85a24"). InnerVolumeSpecName "kube-api-access-f2sfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:21.707141 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:21.707042    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2sfh\" (UniqueName: \"kubernetes.io/projected/63cb0d01-a05c-490e-b069-8ce15ed85a24-kube-api-access-f2sfh\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:22.342394 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:22.342365    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"
Nov 03 23:00:22.342574 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:22.342390    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v" event={"ID":"63cb0d01-a05c-490e-b069-8ce15ed85a24","Type":"ContainerDied","Data":"46c54610705dc9ca878fc73ad0235c0179282c48659c8d6e0bb93c832fb1a7aa"}
Nov 03 23:00:22.342574 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:22.342453    5647 scope.go:122] "RemoveContainer" containerID="8bf648cd7d5a0489af5df3ad9aec259ed7b8db7209992f50f242aca6e1b246f4"
Nov 03 23:00:22.573114 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:22.573075    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"]
Nov 03 23:00:22.576387 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:22.576359    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-924fba-29895780-tw55v"]
Nov 03 23:00:22.700282 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:22.700192    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" path="/var/lib/kubelet/pods/63cb0d01-a05c-490e-b069-8ce15ed85a24/volumes"
Nov 03 23:00:24.770700 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:24.770632    5647 patch_prober.go:29] interesting pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:00:24.772002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:24.771206    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:00:25.254962 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:25.254920    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"]
Nov 03 23:00:25.255234 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:25.255208    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy" containerID="cri-o://d24d865d32e3eec1bd899f669185b8998c3198c5bb279f55302685c1bb121eab" gracePeriod=30
Nov 03 23:00:25.455690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:25.448106    5647 generic.go:356] "Generic (PLEG): container finished" podID="bdd92147-66a8-49de-9612-468a9c71cac1" containerID="497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e" exitCode=0
Nov 03 23:00:25.455690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:25.448291    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" event={"ID":"bdd92147-66a8-49de-9612-468a9c71cac1","Type":"ContainerDied","Data":"497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e"}
Nov 03 23:00:25.455690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:25.448859    5647 scope.go:122] "RemoveContainer" containerID="497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e"
Nov 03 23:00:25.455690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:25.453927    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:00:25.455690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:25.454002    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:00:25.455690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:25.454042    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:00:26.457247 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:26.457209    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" event={"ID":"bdd92147-66a8-49de-9612-468a9c71cac1","Type":"ContainerStarted","Data":"5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3"}
Nov 03 23:00:26.681695 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:26.679904    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 23:00:26.681695 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:26.680169    5647 patch_prober.go:29] interesting pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 03 23:00:26.681695 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:26.680218    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 03 23:00:27.685703 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:27.678309    5647 patch_prober.go:29] interesting pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 03 23:00:27.685703 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:27.678365    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 03 23:00:28.691695 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:28.682108    5647 patch_prober.go:29] interesting pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 03 23:00:28.691695 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:28.682171    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 03 23:00:29.690734 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:29.688759    5647 patch_prober.go:29] interesting pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:00:29.690734 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:29.688837    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:00:29.690734 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:29.689453    5647 patch_prober.go:29] interesting pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 03 23:00:29.690734 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:29.689497    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 03 23:00:30.524765 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.524303    5647 generic.go:356] "Generic (PLEG): container finished" podID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerID="d24d865d32e3eec1bd899f669185b8998c3198c5bb279f55302685c1bb121eab" exitCode=0
Nov 03 23:00:30.524765 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.524437    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" event={"ID":"a0e0cb9a-b472-4d06-89f6-725712c83b89","Type":"ContainerDied","Data":"d24d865d32e3eec1bd899f669185b8998c3198c5bb279f55302685c1bb121eab"}
Nov 03 23:00:30.611448 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.611412    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 23:00:30.712693 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.680088    5647 patch_prober.go:29] interesting pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 03 23:00:30.713049 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.713012    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 03 23:00:30.745702 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.741226    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-certs\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.745702 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.741285    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-podinfo\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.745702 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.741402    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-socket\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.745702 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.741506    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-envoy\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.745702 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.741651    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-data\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.757332 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.752615    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-credential-socket\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.757332 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.752724    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0e0cb9a-b472-4d06-89f6-725712c83b89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0e0cb9a-b472-4d06-89f6-725712c83b89-istiod-ca-cert\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.757332 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.752753    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-kube-api-access-gtvfg\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-kube-api-access-gtvfg\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.757332 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.752819    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-token\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-token\") pod \"a0e0cb9a-b472-4d06-89f6-725712c83b89\" (UID: \"a0e0cb9a-b472-4d06-89f6-725712c83b89\") "
Nov 03 23:00:30.772650 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.772476    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-data" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:30.773202 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.772908    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-socket" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:30.774295 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.774271    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0e0cb9a-b472-4d06-89f6-725712c83b89-istiod-ca-cert" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:30.774683 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.774643    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-credential-socket" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:30.775362 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.775298    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-certs" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:30.832126 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.832075    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-token" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:30.832872 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.832833    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-envoy" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:00:30.833818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.833793    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-podinfo" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:00:30.834492 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.834458    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-kube-api-access-gtvfg" pod "a0e0cb9a-b472-4d06-89f6-725712c83b89" (UID: "a0e0cb9a-b472-4d06-89f6-725712c83b89"). InnerVolumeSpecName "kube-api-access-gtvfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:30.857005 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.856977    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:30.857172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.857158    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:30.857264 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.857251    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:30.857416 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.857404    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0e0cb9a-b472-4d06-89f6-725712c83b89-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:30.857509 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.857498    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtvfg\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-kube-api-access-gtvfg\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:30.857607 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.857595    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:30.857719 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.857705    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:30.857815 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.857803    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0e0cb9a-b472-4d06-89f6-725712c83b89-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:30.857905 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:30.857892    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0e0cb9a-b472-4d06-89f6-725712c83b89-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:31.649698 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.649201    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"
Nov 03 23:00:31.653732 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.651317    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz" event={"ID":"a0e0cb9a-b472-4d06-89f6-725712c83b89","Type":"ContainerDied","Data":"d7cab3a483aaa102b98086f7680553675b2a22edfdd24398a6670efee85aa7c7"}
Nov 03 23:00:31.653732 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.651388    5647 scope.go:122] "RemoveContainer" containerID="d24d865d32e3eec1bd899f669185b8998c3198c5bb279f55302685c1bb121eab"
Nov 03 23:00:31.697736 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.676437    5647 patch_prober.go:29] interesting pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 03 23:00:31.697736 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.676496    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 03 23:00:31.846956 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.846919    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"]
Nov 03 23:00:31.926952 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.926862    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz"]
Nov 03 23:00:31.928362 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.928321    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"]
Nov 03 23:00:31.929574 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929544    5647 cpu_manager.go:414] "containerMap: removing container" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:31.929574 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929573    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929603    5647 cpu_manager.go:414] "containerMap: removing container" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929613    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929649    5647 cpu_manager.go:414] "containerMap: removing container" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929657    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929693    5647 cpu_manager.go:414] "containerMap: removing container" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929700    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929713    5647 cpu_manager.go:414] "containerMap: removing container" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929720    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929731    5647 cpu_manager.go:414] "containerMap: removing container" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929737    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929751    5647 cpu_manager.go:414] "containerMap: removing container" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929757    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929769    5647 cpu_manager.go:414] "containerMap: removing container" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929776    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929791    5647 cpu_manager.go:414] "containerMap: removing container" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929799    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929810    5647 cpu_manager.go:414] "containerMap: removing container" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:31.929816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929816    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929841    5647 cpu_manager.go:414] "containerMap: removing container" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929848    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929863    5647 cpu_manager.go:414] "containerMap: removing container" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929870    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929891    5647 cpu_manager.go:414] "containerMap: removing container" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929898    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929908    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929915    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929929    5647 cpu_manager.go:414] "containerMap: removing container" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929936    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929947    5647 cpu_manager.go:414] "containerMap: removing container" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.929954    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930136    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930150    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930160    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930168    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930180    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930189    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930201    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930212    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930221    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930229    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930240    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930251    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" containerName="istio-proxy"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930262    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="acb7d883-7387-46fe-b8ef-d753e9cee50b" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930273    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="63cb0d01-a05c-490e-b069-8ce15ed85a24" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930282    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:31.930801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.930295    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:31.941794 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.936563    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:31.996696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.987494    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"]
Nov 03 23:00:31.996696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.989314    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-dcf5a5-dockercfg-7svth\""
Nov 03 23:00:31.996696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:31.989441    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-dcf5a5\""
Nov 03 23:00:32.035253 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.029038    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5\" (UID: \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:32.035253 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.029201    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjd4f\" (UniqueName: \"kubernetes.io/projected/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-kube-api-access-kjd4f\") pod \"payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5\" (UID: \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:32.132782 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.132619    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5\" (UID: \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:32.132782 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.132783    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjd4f\" (UniqueName: \"kubernetes.io/projected/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-kube-api-access-kjd4f\") pod \"payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5\" (UID: \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:32.133813 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.133783    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5\" (UID: \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:32.194426 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.194355    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjd4f\" (UniqueName: \"kubernetes.io/projected/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-kube-api-access-kjd4f\") pod \"payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5\" (UID: \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:32.288959 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.288925    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:32.672439 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.672414    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"]
Nov 03 23:00:32.673740 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:00:32.673717    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4940fe_818b_4b7a_a64d_33f8a8c0e954.slice/crio-377c0f7e753afbded65304d914eb22a0d948827a7705257faeb066a31d815f21 WatchSource:0}: Error finding container 377c0f7e753afbded65304d914eb22a0d948827a7705257faeb066a31d815f21: Status 404 returned error can't find the container with id 377c0f7e753afbded65304d914eb22a0d948827a7705257faeb066a31d815f21
Nov 03 23:00:32.681061 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.681037    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 23:00:32.681477 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.681454    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 23:00:32.682134 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.682112    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 23:00:32.700081 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:32.700051    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0e0cb9a-b472-4d06-89f6-725712c83b89" path="/var/lib/kubelet/pods/a0e0cb9a-b472-4d06-89f6-725712c83b89/volumes"
Nov 03 23:00:33.667732 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:33.667697    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5" event={"ID":"4e4940fe-818b-4b7a-a64d-33f8a8c0e954","Type":"ContainerStarted","Data":"d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801"}
Nov 03 23:00:33.667898 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:33.667755    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5" event={"ID":"4e4940fe-818b-4b7a-a64d-33f8a8c0e954","Type":"ContainerStarted","Data":"377c0f7e753afbded65304d914eb22a0d948827a7705257faeb066a31d815f21"}
Nov 03 23:00:33.786065 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:33.786006    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5" podStartSLOduration=2.785991872 podStartE2EDuration="2.785991872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:33.783561124 +0000 UTC m=+2257.224098383" watchObservedRunningTime="2026-11-03 23:00:33.785991872 +0000 UTC m=+2257.226529130"
Nov 03 23:00:34.674600 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:34.674571    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:41.889608 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:41.889571    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"]
Nov 03 23:00:41.890014 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:41.889858    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j" podUID="ea694015-0f0b-494c-9855-9146c5b83c7f" containerName="maas-api" containerID="cri-o://f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76" gracePeriod=30
Nov 03 23:00:42.252357 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.252320    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 23:00:42.303692 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.298274    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-metrics-certs\") pod \"ea694015-0f0b-494c-9855-9146c5b83c7f\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") "
Nov 03 23:00:42.303692 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.298396    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea694015-0f0b-494c-9855-9146c5b83c7f-kube-api-access-gspkv\" (UniqueName: \"kubernetes.io/projected/ea694015-0f0b-494c-9855-9146c5b83c7f-kube-api-access-gspkv\") pod \"ea694015-0f0b-494c-9855-9146c5b83c7f\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") "
Nov 03 23:00:42.303692 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.298437    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls\") pod \"ea694015-0f0b-494c-9855-9146c5b83c7f\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") "
Nov 03 23:00:42.303692 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.298542    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea694015-0f0b-494c-9855-9146c5b83c7f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea694015-0f0b-494c-9855-9146c5b83c7f-openshift-service-ca\") pod \"ea694015-0f0b-494c-9855-9146c5b83c7f\" (UID: \"ea694015-0f0b-494c-9855-9146c5b83c7f\") "
Nov 03 23:00:42.343117 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.343061    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea694015-0f0b-494c-9855-9146c5b83c7f-kube-api-access-gspkv" pod "ea694015-0f0b-494c-9855-9146c5b83c7f" (UID: "ea694015-0f0b-494c-9855-9146c5b83c7f"). InnerVolumeSpecName "kube-api-access-gspkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:00:42.343922 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.343891    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea694015-0f0b-494c-9855-9146c5b83c7f-openshift-service-ca" pod "ea694015-0f0b-494c-9855-9146c5b83c7f" (UID: "ea694015-0f0b-494c-9855-9146c5b83c7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:00:42.347859 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.347813    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls" pod "ea694015-0f0b-494c-9855-9146c5b83c7f" (UID: "ea694015-0f0b-494c-9855-9146c5b83c7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:00:42.348056 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.347894    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-metrics-certs" pod "ea694015-0f0b-494c-9855-9146c5b83c7f" (UID: "ea694015-0f0b-494c-9855-9146c5b83c7f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:00:42.405597 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.405557    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:42.405597 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.405601    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gspkv\" (UniqueName: \"kubernetes.io/projected/ea694015-0f0b-494c-9855-9146c5b83c7f-kube-api-access-gspkv\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:42.405915 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.405621    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea694015-0f0b-494c-9855-9146c5b83c7f-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:42.405915 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.405636    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea694015-0f0b-494c-9855-9146c5b83c7f-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:00:42.731822 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.731790    5647 generic.go:356] "Generic (PLEG): container finished" podID="ea694015-0f0b-494c-9855-9146c5b83c7f" containerID="f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76" exitCode=0
Nov 03 23:00:42.732013 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.731841    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"
Nov 03 23:00:42.732013 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.731871    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j" event={"ID":"ea694015-0f0b-494c-9855-9146c5b83c7f","Type":"ContainerDied","Data":"f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76"}
Nov 03 23:00:42.732013 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.731924    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j" event={"ID":"ea694015-0f0b-494c-9855-9146c5b83c7f","Type":"ContainerDied","Data":"8b4b2c2e8305196f49270c90975cb7b570ff3fa946c3b0707aa922541ea68469"}
Nov 03 23:00:42.732013 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.731944    5647 scope.go:122] "RemoveContainer" containerID="f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76"
Nov 03 23:00:42.747150 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.747105    5647 scope.go:122] "RemoveContainer" containerID="f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76"
Nov 03 23:00:42.747477 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:42.747452    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76\": container with ID starting with f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76 not found: ID does not exist" containerID="f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76"
Nov 03 23:00:42.747563 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.747491    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76"} err="failed to get container status \"f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76\": rpc error: code = NotFound desc = could not find container \"f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76\": container with ID starting with f279aff019e0d4197e5039002aaf616454bc70a304911dc74abee71d66685c76 not found: ID does not exist"
Nov 03 23:00:42.948848 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.948802    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"]
Nov 03 23:00:42.993769 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:42.990356    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9ab23c4-6699f6cc95-wjc8j"]
Nov 03 23:00:43.531852 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.531809    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"]
Nov 03 23:00:43.532537 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532523    5647 cpu_manager.go:414] "containerMap: removing container" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:43.532579 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532539    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:43.532579 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532553    5647 cpu_manager.go:414] "containerMap: removing container" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:43.532579 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532558    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21d23630-c060-406d-b44e-68e52c9930fe" containerName="cleanup"
Nov 03 23:00:43.532579 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532567    5647 cpu_manager.go:414] "containerMap: removing container" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:43.532579 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532573    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:43.532780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532582    5647 cpu_manager.go:414] "containerMap: removing container" podUID="ea694015-0f0b-494c-9855-9146c5b83c7f" containerName="maas-api"
Nov 03 23:00:43.532780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532587    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea694015-0f0b-494c-9855-9146c5b83c7f" containerName="maas-api"
Nov 03 23:00:43.532780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532723    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="14400ebb-956e-45bc-94be-547e4aaf2cfa" containerName="cleanup"
Nov 03 23:00:43.532780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532738    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="c10abf5b-cf83-42b3-9f39-3814244a2422" containerName="cleanup"
Nov 03 23:00:43.532780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532746    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="17f8f446-d9af-4267-bda3-08a01ec2cfc1" containerName="cleanup"
Nov 03 23:00:43.532780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.532754    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea694015-0f0b-494c-9855-9146c5b83c7f" containerName="maas-api"
Nov 03 23:00:43.535999 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.535978    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.591053 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.590923    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1048fc-serving-cert\""
Nov 03 23:00:43.636897 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.636834    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-metrics-certs\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.637170 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.637059    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b03046d-8c44-4dce-9288-f252151ec2e1-openshift-service-ca\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.637170 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.637098    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.637170 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.637165    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zqst\" (UniqueName: \"kubernetes.io/projected/2b03046d-8c44-4dce-9288-f252151ec2e1-kube-api-access-2zqst\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.646625 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.646595    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"]
Nov 03 23:00:43.738316 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.738280    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-metrics-certs\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.738526 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.738443    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b03046d-8c44-4dce-9288-f252151ec2e1-openshift-service-ca\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.738526 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.738467    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.738682 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.738531    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zqst\" (UniqueName: \"kubernetes.io/projected/2b03046d-8c44-4dce-9288-f252151ec2e1-kube-api-access-2zqst\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.738682 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:43.738650    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-serving-cert: secret "maas-api-e2e-api-a-1048fc-serving-cert" not found
Nov 03 23:00:43.738788 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:43.738737    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls podName:2b03046d-8c44-4dce-9288-f252151ec2e1 nodeName:}" failed. No retries permitted until 2026-11-03 23:00:44.238719928 +0000 UTC m=+2267.679257175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls") pod "maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr" (UID: "2b03046d-8c44-4dce-9288-f252151ec2e1") : secret "maas-api-e2e-api-a-1048fc-serving-cert" not found
Nov 03 23:00:43.739198 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.739175    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b03046d-8c44-4dce-9288-f252151ec2e1-openshift-service-ca\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.741802 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.741783    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-metrics-certs\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:43.823746 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:43.816392    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zqst\" (UniqueName: \"kubernetes.io/projected/2b03046d-8c44-4dce-9288-f252151ec2e1-kube-api-access-2zqst\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:44.249037 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:44.248998    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:44.251717 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:44.251691    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls\") pod \"maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:44.449919 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:44.449888    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:44.697651 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:44.697619    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea694015-0f0b-494c-9855-9146c5b83c7f" path="/var/lib/kubelet/pods/ea694015-0f0b-494c-9855-9146c5b83c7f/volumes"
Nov 03 23:00:44.841283 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:44.841251    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"]
Nov 03 23:00:44.842477 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:00:44.842445    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b03046d_8c44_4dce_9288_f252151ec2e1.slice/crio-bf4f788a896a12fc56a2059675ec5a1923764fe472cc52084a742f5ca17c342a WatchSource:0}: Error finding container bf4f788a896a12fc56a2059675ec5a1923764fe472cc52084a742f5ca17c342a: Status 404 returned error can't find the container with id bf4f788a896a12fc56a2059675ec5a1923764fe472cc52084a742f5ca17c342a
Nov 03 23:00:45.684932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:45.684903    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:00:45.814773 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:45.814731    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr" event={"ID":"2b03046d-8c44-4dce-9288-f252151ec2e1","Type":"ContainerStarted","Data":"d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da"}
Nov 03 23:00:45.814977 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:45.814800    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr" event={"ID":"2b03046d-8c44-4dce-9288-f252151ec2e1","Type":"ContainerStarted","Data":"bf4f788a896a12fc56a2059675ec5a1923764fe472cc52084a742f5ca17c342a"}
Nov 03 23:00:45.816846 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:45.816803    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:45.934555 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:45.934448    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr" podStartSLOduration=2.118165725 podStartE2EDuration="2.934433056s" totalImagesPullingTime="816.267331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:00:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:00:45.933159511 +0000 UTC m=+2269.373696769" watchObservedRunningTime="2026-11-03 23:00:45.934433056 +0000 UTC m=+2269.374970315"
Nov 03 23:00:46.075696 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:00:46.069873    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:00:48.541274 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:48.541232    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:00:52.842661 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:52.842627    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:00:56.601092 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:56.601048    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"]
Nov 03 23:00:56.601533 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:56.601292    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy" containerID="cri-o://9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73" gracePeriod=30
Nov 03 23:00:59.889468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:59.889382    5647 patch_prober.go:29] interesting pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:00:59.889468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:00:59.889440    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:01:01.881940 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.881875    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 23:01:01.964128 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.964093    5647 generic.go:356] "Generic (PLEG): container finished" podID="2e63018e-6505-4781-91e1-d261b2acafbc" containerID="9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73" exitCode=0
Nov 03 23:01:01.964364 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.964153    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"
Nov 03 23:01:01.964364 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.964191    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" event={"ID":"2e63018e-6505-4781-91e1-d261b2acafbc","Type":"ContainerDied","Data":"9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73"}
Nov 03 23:01:01.964364 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.964241    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf" event={"ID":"2e63018e-6505-4781-91e1-d261b2acafbc","Type":"ContainerDied","Data":"17796267053c3b215c90bb59cefd8dfbb18280cba9e9c9d6c82cc38b65909243"}
Nov 03 23:01:01.964364 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.964263    5647 scope.go:122] "RemoveContainer" containerID="9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73"
Nov 03 23:01:01.975342 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.975319    5647 scope.go:122] "RemoveContainer" containerID="f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b"
Nov 03 23:01:01.986010 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.985983    5647 scope.go:122] "RemoveContainer" containerID="9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73"
Nov 03 23:01:01.986319 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:01:01.986292    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73\": container with ID starting with 9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73 not found: ID does not exist" containerID="9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73"
Nov 03 23:01:01.986408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.986331    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73"} err="failed to get container status \"9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73\": rpc error: code = NotFound desc = could not find container \"9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73\": container with ID starting with 9941de0c22bd73f108d02fd41eeba386e1cad78fe538ccbd810efa09b01f1c73 not found: ID does not exist"
Nov 03 23:01:01.986408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.986353    5647 scope.go:122] "RemoveContainer" containerID="f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b"
Nov 03 23:01:01.986646 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:01:01.986623    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b\": container with ID starting with f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b not found: ID does not exist" containerID="f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b"
Nov 03 23:01:01.986738 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:01.986657    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b"} err="failed to get container status \"f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b\": rpc error: code = NotFound desc = could not find container \"f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b\": container with ID starting with f854c6ea185b400586bf436936ac9f5fcdb567a23c7c3f0bbbd863bc915e981b not found: ID does not exist"
Nov 03 23:01:02.001061 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001016    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-kube-api-access-jvv55\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-kube-api-access-jvv55\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.001061 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001063    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-certs\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.001340 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001113    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e63018e-6505-4781-91e1-d261b2acafbc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e63018e-6505-4781-91e1-d261b2acafbc-istiod-ca-cert\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.001340 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001173    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e63018e-6505-4781-91e1-d261b2acafbc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e63018e-6505-4781-91e1-d261b2acafbc-istio-podinfo\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.001340 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001236    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-data\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.001340 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001302    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-istio-token\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-istio-token\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.001542 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001360    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-socket\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.001542 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001423    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-envoy\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.001726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001634    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-certs" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:02.002828 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.001834    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-credential-socket\") pod \"2e63018e-6505-4781-91e1-d261b2acafbc\" (UID: \"2e63018e-6505-4781-91e1-d261b2acafbc\") "
Nov 03 23:01:02.002828 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.002638    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.002828 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.002635    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-data" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:02.003074 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.002960    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e63018e-6505-4781-91e1-d261b2acafbc-istiod-ca-cert" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:01:02.003766 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.003368    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-credential-socket" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:02.005158 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.005054    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-socket" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:02.005757 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.005733    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e63018e-6505-4781-91e1-d261b2acafbc-istio-podinfo" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:01:02.005853 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.005768    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-istio-token" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:02.005853 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.005801    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-kube-api-access-jvv55" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "kube-api-access-jvv55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:02.007544 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.007518    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-envoy" pod "2e63018e-6505-4781-91e1-d261b2acafbc" (UID: "2e63018e-6505-4781-91e1-d261b2acafbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:02.104565 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.104519    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.104565 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.104556    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.104565 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.104570    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.104886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.104581    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.104886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.104594    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvv55\" (UniqueName: \"kubernetes.io/projected/2e63018e-6505-4781-91e1-d261b2acafbc-kube-api-access-jvv55\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.104886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.104606    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e63018e-6505-4781-91e1-d261b2acafbc-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.104886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.104617    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e63018e-6505-4781-91e1-d261b2acafbc-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.104886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.104628    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e63018e-6505-4781-91e1-d261b2acafbc-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:02.391290 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.391249    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"]
Nov 03 23:01:02.450280 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.450237    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf"]
Nov 03 23:01:02.717747 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.717625    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" path="/var/lib/kubelet/pods/2e63018e-6505-4781-91e1-d261b2acafbc/volumes"
Nov 03 23:01:02.877478 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.877445    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"]
Nov 03 23:01:02.878215 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.878198    5647 cpu_manager.go:414] "containerMap: removing container" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy"
Nov 03 23:01:02.878273 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.878218    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy"
Nov 03 23:01:02.878367 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.878358    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy"
Nov 03 23:01:02.883430 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.883406    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:02.939154 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.939115    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-78be2c-openshift-default-dockercfg-wr4ff\""
Nov 03 23:01:02.943231 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:02.943191    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"]
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.033869    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-istio-token\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.033932    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-socket\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.033965    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-data\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.034018    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-envoy\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.034213    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-credential-socket\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.034281    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d206c63-b944-4b5b-8952-102b185b6925-istio-podinfo\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.034339    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d206c63-b944-4b5b-8952-102b185b6925-istiod-ca-cert\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.034407    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkkq6\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-kube-api-access-vkkq6\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.043748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.034476    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-certs\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136280 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136245    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-istio-token\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136280 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136296    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-socket\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136280 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136322    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-data\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136721 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136367    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-envoy\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136721 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136501    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-credential-socket\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136721 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136547    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d206c63-b944-4b5b-8952-102b185b6925-istio-podinfo\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136721 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136573    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d206c63-b944-4b5b-8952-102b185b6925-istiod-ca-cert\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136721 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136632    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkkq6\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-kube-api-access-vkkq6\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.136721 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136719    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-certs\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.137136 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136950    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-socket\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.137136 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.136983    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-credential-socket\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.137136 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.137103    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-data\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.137698 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.137614    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-certs\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.138121 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.138073    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d206c63-b944-4b5b-8952-102b185b6925-istiod-ca-cert\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.139833 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.139812    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-envoy\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.140395 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.140376    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d206c63-b944-4b5b-8952-102b185b6925-istio-podinfo\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.208447 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.206100    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-istio-token\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.208447 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.206828    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkkq6\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-kube-api-access-vkkq6\") pod \"e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") " pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.498001 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.497965    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:03.932291 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.932264    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"]
Nov 03 23:01:03.933209 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:01:03.933177    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d206c63_b944_4b5b_8952_102b185b6925.slice/crio-278542c1353af30a377d625020337417be411a00c0f3466d2513ace5fee6015f WatchSource:0}: Error finding container 278542c1353af30a377d625020337417be411a00c0f3466d2513ace5fee6015f: Status 404 returned error can't find the container with id 278542c1353af30a377d625020337417be411a00c0f3466d2513ace5fee6015f
Nov 03 23:01:03.936004 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.935974    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:01:03.936080 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.936052    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:01:03.936127 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.936083    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:01:03.986402 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:03.986368    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" event={"ID":"4d206c63-b944-4b5b-8952-102b185b6925","Type":"ContainerStarted","Data":"278542c1353af30a377d625020337417be411a00c0f3466d2513ace5fee6015f"}
Nov 03 23:01:05.003656 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:05.003609    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" event={"ID":"4d206c63-b944-4b5b-8952-102b185b6925","Type":"ContainerStarted","Data":"8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7"}
Nov 03 23:01:05.224817 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:05.224753    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podStartSLOduration=3.224736346 podStartE2EDuration="3.224736346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:01:05.173011873 +0000 UTC m=+2288.613549133" watchObservedRunningTime="2026-11-03 23:01:05.224736346 +0000 UTC m=+2288.665273604"
Nov 03 23:01:05.498735 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:05.498697    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:05.504488 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:05.504461    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:06.011260 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:06.011226    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:06.012208 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:06.012191    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:01:16.044576 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:01:16.044538    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:01:22.455903 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:22.455860    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:01:32.750504 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:32.750467    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"]
Nov 03 23:01:32.750913 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:32.750771    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" containerID="cri-o://18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af" gracePeriod=30
Nov 03 23:01:32.754156 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:32.754129    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"]
Nov 03 23:01:32.754497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:32.754453    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" containerID="cri-o://fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955" gracePeriod=30
Nov 03 23:01:32.798018 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:32.797984    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"]
Nov 03 23:01:32.798264 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:32.798226    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy" containerID="cri-o://85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97" gracePeriod=30
Nov 03 23:01:34.430745 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:34.430709    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:01:34.431125 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:34.430777    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:01:34.456847 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:34.456805    5647 patch_prober.go:29] interesting pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:01:34.457015 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:34.456866    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:01:38.083396 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.083360    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 23:01:38.130475 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.130437    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 23:01:38.133652 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.133633    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 23:01:38.198088 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198057    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-credential-socket\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.198298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198113    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a163038-fb12-44bf-9948-af32bee1357e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a163038-fb12-44bf-9948-af32bee1357e-istiod-ca-cert\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.198298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198141    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-data\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.198298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198173    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-socket\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.198298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198198    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istiod-ca-cert\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.198298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198228    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-socket\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.198298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198296    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-certs\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198350    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-certs\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198385    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-kube-api-access-x78xx\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-kube-api-access-x78xx\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198400    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-credential-socket" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198412    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-envoy\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198444    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-istio-token\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-istio-token\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198435    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-socket" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198458    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-data" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198520    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-credential-socket\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198599    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-data\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.198629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198632    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-podinfo\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198656    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-socket\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198701    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-certs" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198720    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-socket" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198727    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-certs" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198740    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-data\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198781    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a163038-fb12-44bf-9948-af32bee1357e-istiod-ca-cert" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198793    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-envoy\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198876    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-token\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-token\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198913    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-credential-socket" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198917    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-token\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-token\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198953    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-data" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.198962    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-podinfo\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199026    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-kube-api-access-5rqrf\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-kube-api-access-5rqrf\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199065    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istiod-ca-cert\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.199214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199076    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istiod-ca-cert" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199719    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-data" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199736    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-socket" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199838    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-credential-socket\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199894    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-certs\") pod \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\" (UID: \"9d65809b-0adb-4d76-b9f5-ca690d30df9b\") "
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199910    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istiod-ca-cert" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199928    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a163038-fb12-44bf-9948-af32bee1357e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a163038-fb12-44bf-9948-af32bee1357e-istio-podinfo\") pod \"1a163038-fb12-44bf-9948-af32bee1357e\" (UID: \"1a163038-fb12-44bf-9948-af32bee1357e\") "
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.199981    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-kube-api-access-zp48b\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-kube-api-access-zp48b\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200022    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-envoy\") pod \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\" (UID: \"83a20cc8-b64c-42d3-8d53-3a2a04f49735\") "
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200088    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-credential-socket" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.200190 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200157    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-certs" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.200765 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200740    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.200830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200766    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a163038-fb12-44bf-9948-af32bee1357e-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.200830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200781    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.200830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200795    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.200830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200809    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.200830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200823    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200837    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200853    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200867    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200881    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200893    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200905    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200917    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200930    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.200943    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.201871 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.201840    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-token" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:38.202021 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.201931    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-kube-api-access-x78xx" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "kube-api-access-x78xx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:38.202236 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.202206    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-envoy" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.203316 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.203292    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-envoy" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.203438 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.203337    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-istio-token" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:38.203438 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.203403    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-token" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:38.203559 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.203474    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-kube-api-access-5rqrf" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "kube-api-access-5rqrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:38.203639 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.203528    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-podinfo" pod "9d65809b-0adb-4d76-b9f5-ca690d30df9b" (UID: "9d65809b-0adb-4d76-b9f5-ca690d30df9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:01:38.204205 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.204181    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a163038-fb12-44bf-9948-af32bee1357e-istio-podinfo" pod "1a163038-fb12-44bf-9948-af32bee1357e" (UID: "1a163038-fb12-44bf-9948-af32bee1357e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:01:38.204554 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.204529    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-kube-api-access-zp48b" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "kube-api-access-zp48b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:01:38.204750 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.204732    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-envoy" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:01:38.204985 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.204968    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-podinfo" pod "83a20cc8-b64c-42d3-8d53-3a2a04f49735" (UID: "83a20cc8-b64c-42d3-8d53-3a2a04f49735"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302248    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302286    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302300    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302313    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302326    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d65809b-0adb-4d76-b9f5-ca690d30df9b-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302338    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rqrf\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-kube-api-access-5rqrf\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302354    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a163038-fb12-44bf-9948-af32bee1357e-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302367    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp48b\" (UniqueName: \"kubernetes.io/projected/83a20cc8-b64c-42d3-8d53-3a2a04f49735-kube-api-access-zp48b\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302380    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83a20cc8-b64c-42d3-8d53-3a2a04f49735-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302394    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x78xx\" (UniqueName: \"kubernetes.io/projected/9d65809b-0adb-4d76-b9f5-ca690d30df9b-kube-api-access-x78xx\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302406    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a163038-fb12-44bf-9948-af32bee1357e-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.303034 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.302419    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a163038-fb12-44bf-9948-af32bee1357e-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:01:38.313299 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.313266    5647 generic.go:356] "Generic (PLEG): container finished" podID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerID="fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955" exitCode=0
Nov 03 23:01:38.313475 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.313333    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"
Nov 03 23:01:38.313475 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.313343    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" event={"ID":"9d65809b-0adb-4d76-b9f5-ca690d30df9b","Type":"ContainerDied","Data":"fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955"}
Nov 03 23:01:38.313475 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.313418    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx" event={"ID":"9d65809b-0adb-4d76-b9f5-ca690d30df9b","Type":"ContainerDied","Data":"8e2338cf4385a6fdd9cfc62b8e55dea6a7d69bf24ffc5aa6dfc18e1e06119ef5"}
Nov 03 23:01:38.313475 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.313451    5647 scope.go:122] "RemoveContainer" containerID="fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955"
Nov 03 23:01:38.323435 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.323407    5647 generic.go:356] "Generic (PLEG): container finished" podID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerID="85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97" exitCode=0
Nov 03 23:01:38.323573 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.323465    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"
Nov 03 23:01:38.323573 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.323504    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" event={"ID":"83a20cc8-b64c-42d3-8d53-3a2a04f49735","Type":"ContainerDied","Data":"85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97"}
Nov 03 23:01:38.323786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.323574    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r" event={"ID":"83a20cc8-b64c-42d3-8d53-3a2a04f49735","Type":"ContainerDied","Data":"11a7314107f0fdf1d1599fe9b6dcd4c3d2fcb8d2cf085006c257da49daf5607e"}
Nov 03 23:01:38.326063 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.326037    5647 generic.go:356] "Generic (PLEG): container finished" podID="1a163038-fb12-44bf-9948-af32bee1357e" containerID="18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af" exitCode=0
Nov 03 23:01:38.326187 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.326145    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"
Nov 03 23:01:38.326261 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.326182    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" event={"ID":"1a163038-fb12-44bf-9948-af32bee1357e","Type":"ContainerDied","Data":"18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af"}
Nov 03 23:01:38.326261 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.326242    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp" event={"ID":"1a163038-fb12-44bf-9948-af32bee1357e","Type":"ContainerDied","Data":"d873f84d5608c3c3a248b148e268ef3d24c85aef353bf57a5cb221f8ea5cd622"}
Nov 03 23:01:38.334621 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.334504    5647 scope.go:122] "RemoveContainer" containerID="fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955"
Nov 03 23:01:38.335464 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:01:38.335441    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955\": container with ID starting with fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955 not found: ID does not exist" containerID="fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955"
Nov 03 23:01:38.335567 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.335474    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955"} err="failed to get container status \"fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955\": rpc error: code = NotFound desc = could not find container \"fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955\": container with ID starting with fff177200f84d8e1c6ca02ab82c0490fb2974ac8920f958db903fb4ccce3b955 not found: ID does not exist"
Nov 03 23:01:38.335567 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.335496    5647 scope.go:122] "RemoveContainer" containerID="85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97"
Nov 03 23:01:38.349880 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.349861    5647 scope.go:122] "RemoveContainer" containerID="dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c"
Nov 03 23:01:38.360145 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.360127    5647 scope.go:122] "RemoveContainer" containerID="85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97"
Nov 03 23:01:38.360415 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:01:38.360397    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97\": container with ID starting with 85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97 not found: ID does not exist" containerID="85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97"
Nov 03 23:01:38.360479 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.360423    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97"} err="failed to get container status \"85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97\": rpc error: code = NotFound desc = could not find container \"85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97\": container with ID starting with 85cc8dce8938e26f204ee965f755ba3b2fd2f634ec27f081f27873c265841f97 not found: ID does not exist"
Nov 03 23:01:38.360479 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.360440    5647 scope.go:122] "RemoveContainer" containerID="dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c"
Nov 03 23:01:38.360724 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:01:38.360704    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c\": container with ID starting with dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c not found: ID does not exist" containerID="dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c"
Nov 03 23:01:38.360777 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.360731    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c"} err="failed to get container status \"dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c\": rpc error: code = NotFound desc = could not find container \"dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c\": container with ID starting with dec14ba4ec6a1ff18f177ee568c472e7a02ef4255a14bd16068135a005a9276c not found: ID does not exist"
Nov 03 23:01:38.360777 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.360746    5647 scope.go:122] "RemoveContainer" containerID="18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af"
Nov 03 23:01:38.370169 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.370150    5647 scope.go:122] "RemoveContainer" containerID="18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af"
Nov 03 23:01:38.370446 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:01:38.370423    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af\": container with ID starting with 18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af not found: ID does not exist" containerID="18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af"
Nov 03 23:01:38.370515 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.370455    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af"} err="failed to get container status \"18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af\": rpc error: code = NotFound desc = could not find container \"18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af\": container with ID starting with 18a862d8e9b8f6db27092fefac68ea12d875806ed3f7791f6ed7d077725878af not found: ID does not exist"
Nov 03 23:01:38.618903 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.618800    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"]
Nov 03 23:01:38.620400 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.620373    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx"]
Nov 03 23:01:38.696710 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.696661    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" path="/var/lib/kubelet/pods/9d65809b-0adb-4d76-b9f5-ca690d30df9b/volumes"
Nov 03 23:01:38.723939 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.723905    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"]
Nov 03 23:01:38.728132 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.728093    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r"]
Nov 03 23:01:38.837275 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.837235    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"]
Nov 03 23:01:38.840369 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:38.840341    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp"]
Nov 03 23:01:40.698154 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:40.698120    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a163038-fb12-44bf-9948-af32bee1357e" path="/var/lib/kubelet/pods/1a163038-fb12-44bf-9948-af32bee1357e/volumes"
Nov 03 23:01:40.698629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:01:40.698613    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" path="/var/lib/kubelet/pods/83a20cc8-b64c-42d3-8d53-3a2a04f49735/volumes"
Nov 03 23:01:46.044401 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:01:46.044355    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:02:16.044372 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:02:16.044279    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:02:23.086873 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:23.086835    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"]
Nov 03 23:02:28.087603 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.087546    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw" podUID="f73fd7b6-a571-4586-bb8b-05b775f5b30c" containerName="payload-processing" containerID="cri-o://daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036" gracePeriod=25
Nov 03 23:02:28.373784 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.373753    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 23:02:28.421050 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.420978    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f73fd7b6-a571-4586-bb8b-05b775f5b30c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f73fd7b6-a571-4586-bb8b-05b775f5b30c-plugins-config-volume\") pod \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\" (UID: \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\") "
Nov 03 23:02:28.421050 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.421036    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f73fd7b6-a571-4586-bb8b-05b775f5b30c-kube-api-access-8jf72\" (UniqueName: \"kubernetes.io/projected/f73fd7b6-a571-4586-bb8b-05b775f5b30c-kube-api-access-8jf72\") pod \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\" (UID: \"f73fd7b6-a571-4586-bb8b-05b775f5b30c\") "
Nov 03 23:02:28.421367 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.421344    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f73fd7b6-a571-4586-bb8b-05b775f5b30c-plugins-config-volume" pod "f73fd7b6-a571-4586-bb8b-05b775f5b30c" (UID: "f73fd7b6-a571-4586-bb8b-05b775f5b30c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:02:28.421638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.421624    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f73fd7b6-a571-4586-bb8b-05b775f5b30c-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:02:28.423464 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.423438    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f73fd7b6-a571-4586-bb8b-05b775f5b30c-kube-api-access-8jf72" pod "f73fd7b6-a571-4586-bb8b-05b775f5b30c" (UID: "f73fd7b6-a571-4586-bb8b-05b775f5b30c"). InnerVolumeSpecName "kube-api-access-8jf72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:02:28.523945 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.523907    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jf72\" (UniqueName: \"kubernetes.io/projected/f73fd7b6-a571-4586-bb8b-05b775f5b30c-kube-api-access-8jf72\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:02:28.931738 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.931662    5647 generic.go:356] "Generic (PLEG): container finished" podID="f73fd7b6-a571-4586-bb8b-05b775f5b30c" containerID="daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036" exitCode=0
Nov 03 23:02:28.931738 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.931734    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"
Nov 03 23:02:28.932023 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.931782    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw" event={"ID":"f73fd7b6-a571-4586-bb8b-05b775f5b30c","Type":"ContainerDied","Data":"daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036"}
Nov 03 23:02:28.932023 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.931838    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw" event={"ID":"f73fd7b6-a571-4586-bb8b-05b775f5b30c","Type":"ContainerDied","Data":"86f329eea6c8b38acd3b51e4a14dd5dea5af978bad0867cb731d29be397a0f0a"}
Nov 03 23:02:28.932023 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.931864    5647 scope.go:122] "RemoveContainer" containerID="daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036"
Nov 03 23:02:28.942428 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.942402    5647 scope.go:122] "RemoveContainer" containerID="daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036"
Nov 03 23:02:28.942746 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:02:28.942723    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036\": container with ID starting with daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036 not found: ID does not exist" containerID="daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036"
Nov 03 23:02:28.942801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:28.942756    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036"} err="failed to get container status \"daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036\": rpc error: code = NotFound desc = could not find container \"daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036\": container with ID starting with daa01826ae3072a6faff1950a46a3f44636e3094b01d164bd61616c0d00f4036 not found: ID does not exist"
Nov 03 23:02:29.096122 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:29.096036    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"]
Nov 03 23:02:29.144949 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:29.144912    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw"]
Nov 03 23:02:30.771697 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.766655    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f73fd7b6-a571-4586-bb8b-05b775f5b30c" path="/var/lib/kubelet/pods/f73fd7b6-a571-4586-bb8b-05b775f5b30c/volumes"
Nov 03 23:02:30.962915 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.962875    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"]
Nov 03 23:02:30.963596 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963581    5647 cpu_manager.go:414] "containerMap: removing container" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy"
Nov 03 23:02:30.963643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963600    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy"
Nov 03 23:02:30.963643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963614    5647 cpu_manager.go:414] "containerMap: removing container" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy"
Nov 03 23:02:30.963643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963621    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy"
Nov 03 23:02:30.963643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963637    5647 cpu_manager.go:414] "containerMap: removing container" podUID="f73fd7b6-a571-4586-bb8b-05b775f5b30c" containerName="payload-processing"
Nov 03 23:02:30.963643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963642    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f73fd7b6-a571-4586-bb8b-05b775f5b30c" containerName="payload-processing"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963657    5647 cpu_manager.go:414] "containerMap: removing container" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963662    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963746    5647 cpu_manager.go:414] "containerMap: removing container" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963752    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963777    5647 cpu_manager.go:414] "containerMap: removing container" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963782    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963906    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d65809b-0adb-4d76-b9f5-ca690d30df9b" containerName="istio-proxy"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963918    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy"
Nov 03 23:02:30.963926 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963928    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="f73fd7b6-a571-4586-bb8b-05b775f5b30c" containerName="payload-processing"
Nov 03 23:02:30.964272 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963940    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="83a20cc8-b64c-42d3-8d53-3a2a04f49735" containerName="istio-proxy"
Nov 03 23:02:30.964272 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963950    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e63018e-6505-4781-91e1-d261b2acafbc" containerName="istio-proxy"
Nov 03 23:02:30.964272 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.963958    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a163038-fb12-44bf-9948-af32bee1357e" containerName="istio-proxy"
Nov 03 23:02:30.969152 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:30.969131    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:31.021373 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.021332    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-cfe1bd-dockercfg-xpgh2\""
Nov 03 23:02:31.027886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.027718    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"]
Nov 03 23:02:31.080821 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.080789    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-cfe1bd\""
Nov 03 23:02:31.205785 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.205748    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"]
Nov 03 23:02:31.224945 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.224542    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"]
Nov 03 23:02:31.224945 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.224734    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:31.293733 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.293620    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njntg\" (UniqueName: \"kubernetes.io/projected/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-kube-api-access-njntg\") pod \"payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk\" (UID: \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:31.293879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.293823    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk\" (UID: \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:31.395271 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.395234    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81b3584-43d1-4ace-91fd-b729b7a0112e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd\" (UID: \"a81b3584-43d1-4ace-91fd-b729b7a0112e\") " pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:31.395446 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.395365    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njntg\" (UniqueName: \"kubernetes.io/projected/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-kube-api-access-njntg\") pod \"payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk\" (UID: \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:31.395446 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.395397    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gvsx\" (UniqueName: \"kubernetes.io/projected/a81b3584-43d1-4ace-91fd-b729b7a0112e-kube-api-access-2gvsx\") pod \"payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd\" (UID: \"a81b3584-43d1-4ace-91fd-b729b7a0112e\") " pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:31.395533 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.395518    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk\" (UID: \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:31.396276 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.396237    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk\" (UID: \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:31.456320 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.456279    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njntg\" (UniqueName: \"kubernetes.io/projected/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-kube-api-access-njntg\") pod \"payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk\" (UID: \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:31.496795 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.496760    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81b3584-43d1-4ace-91fd-b729b7a0112e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd\" (UID: \"a81b3584-43d1-4ace-91fd-b729b7a0112e\") " pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:31.496973 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.496858    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvsx\" (UniqueName: \"kubernetes.io/projected/a81b3584-43d1-4ace-91fd-b729b7a0112e-kube-api-access-2gvsx\") pod \"payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd\" (UID: \"a81b3584-43d1-4ace-91fd-b729b7a0112e\") " pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:31.497476 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.497455    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81b3584-43d1-4ace-91fd-b729b7a0112e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd\" (UID: \"a81b3584-43d1-4ace-91fd-b729b7a0112e\") " pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:31.557842 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.557764    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gvsx\" (UniqueName: \"kubernetes.io/projected/a81b3584-43d1-4ace-91fd-b729b7a0112e-kube-api-access-2gvsx\") pod \"payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd\" (UID: \"a81b3584-43d1-4ace-91fd-b729b7a0112e\") " pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:31.584088 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.584057    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:31.845390 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.845301    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:31.966194 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:31.966158    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"]
Nov 03 23:02:31.967526 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:02:31.967490    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf69f6de_21e1_40a8_a9cd_5e8a22e57ee7.slice/crio-63f608a5486f32a7fed1ccadc204d30ca91d88739879f1a0ad4f279e02c29d50 WatchSource:0}: Error finding container 63f608a5486f32a7fed1ccadc204d30ca91d88739879f1a0ad4f279e02c29d50: Status 404 returned error can't find the container with id 63f608a5486f32a7fed1ccadc204d30ca91d88739879f1a0ad4f279e02c29d50
Nov 03 23:02:32.229608 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:32.229582    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"]
Nov 03 23:02:32.230263 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:02:32.230236    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81b3584_43d1_4ace_91fd_b729b7a0112e.slice/crio-bb5362feca703942f964d494c7fdcb1a68b6b5f1f8e148de24a39350afcf2185 WatchSource:0}: Error finding container bb5362feca703942f964d494c7fdcb1a68b6b5f1f8e148de24a39350afcf2185: Status 404 returned error can't find the container with id bb5362feca703942f964d494c7fdcb1a68b6b5f1f8e148de24a39350afcf2185
Nov 03 23:02:32.967011 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:32.966977    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk" event={"ID":"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7","Type":"ContainerStarted","Data":"95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670"}
Nov 03 23:02:32.967481 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:32.967044    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:32.967481 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:32.967071    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk" event={"ID":"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7","Type":"ContainerStarted","Data":"63f608a5486f32a7fed1ccadc204d30ca91d88739879f1a0ad4f279e02c29d50"}
Nov 03 23:02:32.969043 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:32.969018    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd" event={"ID":"a81b3584-43d1-4ace-91fd-b729b7a0112e","Type":"ContainerStarted","Data":"0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8"}
Nov 03 23:02:32.969172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:32.969067    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd" event={"ID":"a81b3584-43d1-4ace-91fd-b729b7a0112e","Type":"ContainerStarted","Data":"bb5362feca703942f964d494c7fdcb1a68b6b5f1f8e148de24a39350afcf2185"}
Nov 03 23:02:32.969172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:32.969104    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:33.082223 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:33.082171    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk" podStartSLOduration=3.082159377 podStartE2EDuration="3.082159377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:02:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:02:33.081119353 +0000 UTC m=+2376.521656612" watchObservedRunningTime="2026-11-03 23:02:33.082159377 +0000 UTC m=+2376.522696635"
Nov 03 23:02:33.237752 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:33.229874    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd" podStartSLOduration=2.229851017 podStartE2EDuration="2.229851017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:02:33.216131544 +0000 UTC m=+2376.656668804" watchObservedRunningTime="2026-11-03 23:02:33.229851017 +0000 UTC m=+2376.670388293"
Nov 03 23:02:38.725518 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:38.725488    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:02:42.573693 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:42.572827    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:02:43.979306 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:43.979276    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:02:43.979908 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:43.979392    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:02:46.106456 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:02:46.106416    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:02:57.068879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:57.068850    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:02:57.076462 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:02:57.076438    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:03:08.482729 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:08.482693    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"]
Nov 03 23:03:08.587811 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:08.587779    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"]
Nov 03 23:03:13.483237 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.483199    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg" podUID="67837717-e0d8-445d-82ae-7392f7a104c3" containerName="payload-processing" containerID="cri-o://0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e" gracePeriod=25
Nov 03 23:03:13.588247 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.588211    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp" podUID="f62143b4-dae9-42b0-a6a1-e042d8e71257" containerName="payload-pre-processing" containerID="cri-o://3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81" gracePeriod=25
Nov 03 23:03:13.764798 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.764771    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 23:03:13.848576 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.848543    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67837717-e0d8-445d-82ae-7392f7a104c3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67837717-e0d8-445d-82ae-7392f7a104c3-plugins-config-volume\") pod \"67837717-e0d8-445d-82ae-7392f7a104c3\" (UID: \"67837717-e0d8-445d-82ae-7392f7a104c3\") "
Nov 03 23:03:13.848789 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.848720    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67837717-e0d8-445d-82ae-7392f7a104c3-kube-api-access-xxftw\" (UniqueName: \"kubernetes.io/projected/67837717-e0d8-445d-82ae-7392f7a104c3-kube-api-access-xxftw\") pod \"67837717-e0d8-445d-82ae-7392f7a104c3\" (UID: \"67837717-e0d8-445d-82ae-7392f7a104c3\") "
Nov 03 23:03:13.848981 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.848953    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67837717-e0d8-445d-82ae-7392f7a104c3-plugins-config-volume" pod "67837717-e0d8-445d-82ae-7392f7a104c3" (UID: "67837717-e0d8-445d-82ae-7392f7a104c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:03:13.849412 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.849392    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67837717-e0d8-445d-82ae-7392f7a104c3-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:13.851511 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.851486    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67837717-e0d8-445d-82ae-7392f7a104c3-kube-api-access-xxftw" pod "67837717-e0d8-445d-82ae-7392f7a104c3" (UID: "67837717-e0d8-445d-82ae-7392f7a104c3"). InnerVolumeSpecName "kube-api-access-xxftw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:13.866136 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.866115    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 23:03:13.950438 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.950408    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f62143b4-dae9-42b0-a6a1-e042d8e71257-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f62143b4-dae9-42b0-a6a1-e042d8e71257-plugins-config-volume\") pod \"f62143b4-dae9-42b0-a6a1-e042d8e71257\" (UID: \"f62143b4-dae9-42b0-a6a1-e042d8e71257\") "
Nov 03 23:03:13.950637 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.950515    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f62143b4-dae9-42b0-a6a1-e042d8e71257-kube-api-access-bbz7x\" (UniqueName: \"kubernetes.io/projected/f62143b4-dae9-42b0-a6a1-e042d8e71257-kube-api-access-bbz7x\") pod \"f62143b4-dae9-42b0-a6a1-e042d8e71257\" (UID: \"f62143b4-dae9-42b0-a6a1-e042d8e71257\") "
Nov 03 23:03:13.950836 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.950813    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f62143b4-dae9-42b0-a6a1-e042d8e71257-plugins-config-volume" pod "f62143b4-dae9-42b0-a6a1-e042d8e71257" (UID: "f62143b4-dae9-42b0-a6a1-e042d8e71257"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:03:13.951061 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.951042    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxftw\" (UniqueName: \"kubernetes.io/projected/67837717-e0d8-445d-82ae-7392f7a104c3-kube-api-access-xxftw\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:13.951061 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.951060    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f62143b4-dae9-42b0-a6a1-e042d8e71257-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:13.952897 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:13.952876    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f62143b4-dae9-42b0-a6a1-e042d8e71257-kube-api-access-bbz7x" pod "f62143b4-dae9-42b0-a6a1-e042d8e71257" (UID: "f62143b4-dae9-42b0-a6a1-e042d8e71257"). InnerVolumeSpecName "kube-api-access-bbz7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:14.052627 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.052536    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbz7x\" (UniqueName: \"kubernetes.io/projected/f62143b4-dae9-42b0-a6a1-e042d8e71257-kube-api-access-bbz7x\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:14.311196 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.311097    5647 generic.go:356] "Generic (PLEG): container finished" podID="67837717-e0d8-445d-82ae-7392f7a104c3" containerID="0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e" exitCode=0
Nov 03 23:03:14.311196 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.311138    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"
Nov 03 23:03:14.311428 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.311192    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg" event={"ID":"67837717-e0d8-445d-82ae-7392f7a104c3","Type":"ContainerDied","Data":"0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e"}
Nov 03 23:03:14.311428 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.311247    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg" event={"ID":"67837717-e0d8-445d-82ae-7392f7a104c3","Type":"ContainerDied","Data":"9e87b2c0bf7622a43c62f4861e075a6f1a18945cdda67d7678c7558cd639aff0"}
Nov 03 23:03:14.311428 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.311270    5647 scope.go:122] "RemoveContainer" containerID="0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e"
Nov 03 23:03:14.312793 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.312766    5647 generic.go:356] "Generic (PLEG): container finished" podID="f62143b4-dae9-42b0-a6a1-e042d8e71257" containerID="3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81" exitCode=0
Nov 03 23:03:14.312793 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.312812    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"
Nov 03 23:03:14.312793 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.312800    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp" event={"ID":"f62143b4-dae9-42b0-a6a1-e042d8e71257","Type":"ContainerDied","Data":"3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81"}
Nov 03 23:03:14.312793 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.312882    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp" event={"ID":"f62143b4-dae9-42b0-a6a1-e042d8e71257","Type":"ContainerDied","Data":"8262744cc470c088ac2cb39e3068e44a93a5a226fcc013511ee8f348ade23264"}
Nov 03 23:03:14.322563 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.322527    5647 scope.go:122] "RemoveContainer" containerID="0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e"
Nov 03 23:03:14.322904 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:03:14.322884    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e\": container with ID starting with 0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e not found: ID does not exist" containerID="0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e"
Nov 03 23:03:14.322986 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.322916    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e"} err="failed to get container status \"0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e\": rpc error: code = NotFound desc = could not find container \"0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e\": container with ID starting with 0ca2ea9d9242d96d9acedf821719f72cb0e12012d77782f4fafe71352a39951e not found: ID does not exist"
Nov 03 23:03:14.322986 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.322934    5647 scope.go:122] "RemoveContainer" containerID="3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81"
Nov 03 23:03:14.334908 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.334890    5647 scope.go:122] "RemoveContainer" containerID="3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81"
Nov 03 23:03:14.335322 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:03:14.335301    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81\": container with ID starting with 3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81 not found: ID does not exist" containerID="3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81"
Nov 03 23:03:14.335376 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.335330    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81"} err="failed to get container status \"3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81\": rpc error: code = NotFound desc = could not find container \"3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81\": container with ID starting with 3b11e3ee9bde5265c38b2b9f7c89b02aa33d467660110effe61317ae1c161a81 not found: ID does not exist"
Nov 03 23:03:14.475598 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.475562    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"]
Nov 03 23:03:14.524808 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.524766    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp"]
Nov 03 23:03:14.632800 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.632646    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"]
Nov 03 23:03:14.637817 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.637790    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg"]
Nov 03 23:03:14.698036 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.698005    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67837717-e0d8-445d-82ae-7392f7a104c3" path="/var/lib/kubelet/pods/67837717-e0d8-445d-82ae-7392f7a104c3/volumes"
Nov 03 23:03:14.698424 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:14.698410    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f62143b4-dae9-42b0-a6a1-e042d8e71257" path="/var/lib/kubelet/pods/f62143b4-dae9-42b0-a6a1-e042d8e71257/volumes"
Nov 03 23:03:16.123697 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:03:16.121208    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:03:28.457907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:28.457876    5647 generic.go:356] "Generic (PLEG): container finished" podID="4d206c63-b944-4b5b-8952-102b185b6925" containerID="8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7" exitCode=0
Nov 03 23:03:28.458343 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:28.457939    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" event={"ID":"4d206c63-b944-4b5b-8952-102b185b6925","Type":"ContainerDied","Data":"8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7"}
Nov 03 23:03:28.458394 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:28.458377    5647 scope.go:122] "RemoveContainer" containerID="8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7"
Nov 03 23:03:28.460064 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:28.460035    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:03:28.460143 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:28.460105    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:03:28.460143 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:28.460136    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:03:29.466628 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:29.466593    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" event={"ID":"4d206c63-b944-4b5b-8952-102b185b6925","Type":"ContainerStarted","Data":"9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0"}
Nov 03 23:03:29.500427 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:29.500389    5647 patch_prober.go:29] interesting pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Nov 03 23:03:29.500655 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:29.500456    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Nov 03 23:03:29.500807 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:29.500782    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:03:30.524701 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:30.515713    5647 patch_prober.go:29] interesting pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Nov 03 23:03:30.524701 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:30.515770    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Nov 03 23:03:31.499131 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:31.499094    5647 patch_prober.go:29] interesting pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Nov 03 23:03:31.499349 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:31.499150    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Nov 03 23:03:32.498651 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:32.498603    5647 patch_prober.go:29] interesting pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Nov 03 23:03:32.499124 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:32.498695    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Nov 03 23:03:33.499247 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:33.499208    5647 patch_prober.go:29] interesting pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Nov 03 23:03:33.499638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:33.499265    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Nov 03 23:03:34.498908 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:34.498871    5647 patch_prober.go:29] interesting pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Nov 03 23:03:34.499105 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:34.498926    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Nov 03 23:03:34.639694 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:34.638487    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"]
Nov 03 23:03:35.503217 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:35.503187    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:03:35.503690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:35.503653    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:03:35.504461 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:35.504443    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:03:39.639437 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:39.639394    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55" podUID="5da35a7f-116b-4518-b27b-4fc35a4c7f9e" containerName="payload-pre-processing" containerID="cri-o://9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee" gracePeriod=25
Nov 03 23:03:39.922842 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:39.922815    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 23:03:40.029786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.029740    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-plugins-config-volume\") pod \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\" (UID: \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\") "
Nov 03 23:03:40.030014 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.029996    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-kube-api-access-lfb7q\" (UniqueName: \"kubernetes.io/projected/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-kube-api-access-lfb7q\") pod \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\" (UID: \"5da35a7f-116b-4518-b27b-4fc35a4c7f9e\") "
Nov 03 23:03:40.030155 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.030128    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-plugins-config-volume" pod "5da35a7f-116b-4518-b27b-4fc35a4c7f9e" (UID: "5da35a7f-116b-4518-b27b-4fc35a4c7f9e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:03:40.030569 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.030548    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:40.032286 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.032266    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-kube-api-access-lfb7q" pod "5da35a7f-116b-4518-b27b-4fc35a4c7f9e" (UID: "5da35a7f-116b-4518-b27b-4fc35a4c7f9e"). InnerVolumeSpecName "kube-api-access-lfb7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:40.132290 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.132258    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfb7q\" (UniqueName: \"kubernetes.io/projected/5da35a7f-116b-4518-b27b-4fc35a4c7f9e-kube-api-access-lfb7q\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:40.607554 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.607513    5647 generic.go:356] "Generic (PLEG): container finished" podID="5da35a7f-116b-4518-b27b-4fc35a4c7f9e" containerID="9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee" exitCode=0
Nov 03 23:03:40.607757 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.607566    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"
Nov 03 23:03:40.607757 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.607605    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55" event={"ID":"5da35a7f-116b-4518-b27b-4fc35a4c7f9e","Type":"ContainerDied","Data":"9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee"}
Nov 03 23:03:40.607757 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.607681    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55" event={"ID":"5da35a7f-116b-4518-b27b-4fc35a4c7f9e","Type":"ContainerDied","Data":"dd91049adeaf14097653ef76ca53fd1da0e2efd07d9412850983d5d41cb4450a"}
Nov 03 23:03:40.607757 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.607705    5647 scope.go:122] "RemoveContainer" containerID="9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee"
Nov 03 23:03:40.619288 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.619268    5647 scope.go:122] "RemoveContainer" containerID="9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee"
Nov 03 23:03:40.619631 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:03:40.619602    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee\": container with ID starting with 9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee not found: ID does not exist" containerID="9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee"
Nov 03 23:03:40.619847 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.619638    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee"} err="failed to get container status \"9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee\": rpc error: code = NotFound desc = could not find container \"9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee\": container with ID starting with 9f68af5e8a4a4ef99e9bcd28ea0bbe60fdba17902d67ac10d2a98f55f53f5eee not found: ID does not exist"
Nov 03 23:03:40.774697 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.774649    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"]
Nov 03 23:03:40.826507 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:40.826462    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55"]
Nov 03 23:03:42.698708 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:42.698628    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5da35a7f-116b-4518-b27b-4fc35a4c7f9e" path="/var/lib/kubelet/pods/5da35a7f-116b-4518-b27b-4fc35a4c7f9e/volumes"
Nov 03 23:03:45.095952 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:45.095916    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"]
Nov 03 23:03:45.145069 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:45.145037    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"]
Nov 03 23:03:46.044560 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:03:46.044521    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:03:53.806425 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:53.806391    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"]
Nov 03 23:03:53.922748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:53.922713    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"]
Nov 03 23:03:54.346652 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:54.346609    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:03:58.806952 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:58.806893    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds" podUID="8576b4d8-ff39-4559-aa81-8570063bfe4a" containerName="payload-processing" containerID="cri-o://4bb3910ea2294d45bb65e17eae2e4f64af2edcc3b58eaf22ce707585da4105c3" gracePeriod=25
Nov 03 23:03:58.923358 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:58.923324    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx" podUID="6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198" containerName="payload-pre-processing" containerID="cri-o://474a49cf561c6d0bef72bd6db96860c2b123a750bf2535ab355887c361ddcdff" gracePeriod=25
Nov 03 23:03:59.107606 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.107574    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 23:03:59.132546 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.132514    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8576b4d8-ff39-4559-aa81-8570063bfe4a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8576b4d8-ff39-4559-aa81-8570063bfe4a-plugins-config-volume\") pod \"8576b4d8-ff39-4559-aa81-8570063bfe4a\" (UID: \"8576b4d8-ff39-4559-aa81-8570063bfe4a\") "
Nov 03 23:03:59.132753 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.132596    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8576b4d8-ff39-4559-aa81-8570063bfe4a-kube-api-access-hxqbg\" (UniqueName: \"kubernetes.io/projected/8576b4d8-ff39-4559-aa81-8570063bfe4a-kube-api-access-hxqbg\") pod \"8576b4d8-ff39-4559-aa81-8570063bfe4a\" (UID: \"8576b4d8-ff39-4559-aa81-8570063bfe4a\") "
Nov 03 23:03:59.133156 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.132892    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8576b4d8-ff39-4559-aa81-8570063bfe4a-plugins-config-volume" pod "8576b4d8-ff39-4559-aa81-8570063bfe4a" (UID: "8576b4d8-ff39-4559-aa81-8570063bfe4a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:03:59.133156 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.133135    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8576b4d8-ff39-4559-aa81-8570063bfe4a-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:59.134950 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.134925    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8576b4d8-ff39-4559-aa81-8570063bfe4a-kube-api-access-hxqbg" pod "8576b4d8-ff39-4559-aa81-8570063bfe4a" (UID: "8576b4d8-ff39-4559-aa81-8570063bfe4a"). InnerVolumeSpecName "kube-api-access-hxqbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:59.181216 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.181189    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 23:03:59.234010 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.233982    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-plugins-config-volume\") pod \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\" (UID: \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\") "
Nov 03 23:03:59.234171 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.234134    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-kube-api-access-zssnm\" (UniqueName: \"kubernetes.io/projected/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-kube-api-access-zssnm\") pod \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\" (UID: \"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198\") "
Nov 03 23:03:59.234339 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.234317    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-plugins-config-volume" pod "6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198" (UID: "6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:03:59.234685 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.234636    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:59.234807 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.234700    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxqbg\" (UniqueName: \"kubernetes.io/projected/8576b4d8-ff39-4559-aa81-8570063bfe4a-kube-api-access-hxqbg\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:59.236526 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.236502    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-kube-api-access-zssnm" pod "6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198" (UID: "6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198"). InnerVolumeSpecName "kube-api-access-zssnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:03:59.336085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.335998    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zssnm\" (UniqueName: \"kubernetes.io/projected/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198-kube-api-access-zssnm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:03:59.683266 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.683187    5647 scope.go:122] "RemoveContainer" containerID="4bb3910ea2294d45bb65e17eae2e4f64af2edcc3b58eaf22ce707585da4105c3"
Nov 03 23:03:59.693556 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.693531    5647 scope.go:122] "RemoveContainer" containerID="474a49cf561c6d0bef72bd6db96860c2b123a750bf2535ab355887c361ddcdff"
Nov 03 23:03:59.742006 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.741978    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds" event={"ID":"8576b4d8-ff39-4559-aa81-8570063bfe4a","Type":"ContainerDied","Data":"4bb3910ea2294d45bb65e17eae2e4f64af2edcc3b58eaf22ce707585da4105c3"}
Nov 03 23:03:59.742120 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.742038    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"
Nov 03 23:03:59.742120 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.742048    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"
Nov 03 23:03:59.742120 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.742037    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds" event={"ID":"8576b4d8-ff39-4559-aa81-8570063bfe4a","Type":"ContainerDied","Data":"fd45ea3599a35aa6ab078331ac18419fea321c213cf8bebfa6584f829e3439dc"}
Nov 03 23:03:59.742261 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.742146    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx" event={"ID":"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198","Type":"ContainerDied","Data":"474a49cf561c6d0bef72bd6db96860c2b123a750bf2535ab355887c361ddcdff"}
Nov 03 23:03:59.742261 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:03:59.742174    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx" event={"ID":"6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198","Type":"ContainerDied","Data":"d88ceed0f6655110127383bb0a1708cfbd0f173c548d06d435ef46d976224ae0"}
Nov 03 23:04:00.031842 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.031812    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"]
Nov 03 23:04:00.036485 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.036456    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx"]
Nov 03 23:04:00.122302 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.122235    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main" containerID="cri-o://b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7" gracePeriod=45
Nov 03 23:04:00.144865 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.144783    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"]
Nov 03 23:04:00.148046 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.148024    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds"]
Nov 03 23:04:00.167896 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.167858    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main" containerID="cri-o://f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d" gracePeriod=45
Nov 03 23:04:00.589295 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.589270    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 23:04:00.591794 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.591773    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 23:04:00.699264 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.699181    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198" path="/var/lib/kubelet/pods/6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198/volumes"
Nov 03 23:04:00.699625 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.699611    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8576b4d8-ff39-4559-aa81-8570063bfe4a" path="/var/lib/kubelet/pods/8576b4d8-ff39-4559-aa81-8570063bfe4a/volumes"
Nov 03 23:04:00.734494 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734465    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-home\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-home\") pod \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") "
Nov 03 23:04:00.734708 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734516    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-kube-api-access-hw49c\" (UniqueName: \"kubernetes.io/projected/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-kube-api-access-hw49c\") pod \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") "
Nov 03 23:04:00.734708 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734547    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44a7b927-3900-4a1f-9270-b72dd3239865-kube-api-access-k7nlz\" (UniqueName: \"kubernetes.io/projected/44a7b927-3900-4a1f-9270-b72dd3239865-kube-api-access-k7nlz\") pod \"44a7b927-3900-4a1f-9270-b72dd3239865\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") "
Nov 03 23:04:00.734708 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734578    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-model-cache\") pod \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") "
Nov 03 23:04:00.734708 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734699    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-home\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-home\") pod \"44a7b927-3900-4a1f-9270-b72dd3239865\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") "
Nov 03 23:04:00.734942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734753    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-model-cache\") pod \"44a7b927-3900-4a1f-9270-b72dd3239865\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") "
Nov 03 23:04:00.734942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734792    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-home" pod "2ba930d7-6e52-45ac-a9c5-95e03546ee2d" (UID: "2ba930d7-6e52-45ac-a9c5-95e03546ee2d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:00.734942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734840    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-dshm\") pod \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") "
Nov 03 23:04:00.734942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734878    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tls-certs\") pod \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") "
Nov 03 23:04:00.735148 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.734962    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44a7b927-3900-4a1f-9270-b72dd3239865-tls-certs\" (UniqueName: \"kubernetes.io/secret/44a7b927-3900-4a1f-9270-b72dd3239865-tls-certs\") pod \"44a7b927-3900-4a1f-9270-b72dd3239865\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") "
Nov 03 23:04:00.735148 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735000    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-dshm\") pod \"44a7b927-3900-4a1f-9270-b72dd3239865\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") "
Nov 03 23:04:00.735148 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735099    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-tmp-dir\") pod \"44a7b927-3900-4a1f-9270-b72dd3239865\" (UID: \"44a7b927-3900-4a1f-9270-b72dd3239865\") "
Nov 03 23:04:00.735148 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735132    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tmp-dir\") pod \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\" (UID: \"2ba930d7-6e52-45ac-a9c5-95e03546ee2d\") "
Nov 03 23:04:00.735343 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735301    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-model-cache" pod "44a7b927-3900-4a1f-9270-b72dd3239865" (UID: "44a7b927-3900-4a1f-9270-b72dd3239865"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:00.735841 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735531    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-model-cache" pod "2ba930d7-6e52-45ac-a9c5-95e03546ee2d" (UID: "2ba930d7-6e52-45ac-a9c5-95e03546ee2d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:00.735841 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735746    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tmp-dir" pod "2ba930d7-6e52-45ac-a9c5-95e03546ee2d" (UID: "2ba930d7-6e52-45ac-a9c5-95e03546ee2d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:00.736037 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735846    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.736037 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735864    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.736037 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.735878    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.736204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.736089    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-home" pod "44a7b927-3900-4a1f-9270-b72dd3239865" (UID: "44a7b927-3900-4a1f-9270-b72dd3239865"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:00.736961 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.736933    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-tmp-dir" pod "44a7b927-3900-4a1f-9270-b72dd3239865" (UID: "44a7b927-3900-4a1f-9270-b72dd3239865"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:00.737907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.737878    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a7b927-3900-4a1f-9270-b72dd3239865-kube-api-access-k7nlz" pod "44a7b927-3900-4a1f-9270-b72dd3239865" (UID: "44a7b927-3900-4a1f-9270-b72dd3239865"). InnerVolumeSpecName "kube-api-access-k7nlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:00.738362 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.738326    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-dshm" pod "2ba930d7-6e52-45ac-a9c5-95e03546ee2d" (UID: "2ba930d7-6e52-45ac-a9c5-95e03546ee2d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:00.738960 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.738933    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-kube-api-access-hw49c" pod "2ba930d7-6e52-45ac-a9c5-95e03546ee2d" (UID: "2ba930d7-6e52-45ac-a9c5-95e03546ee2d"). InnerVolumeSpecName "kube-api-access-hw49c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:00.739179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.739144    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44a7b927-3900-4a1f-9270-b72dd3239865-tls-certs" pod "44a7b927-3900-4a1f-9270-b72dd3239865" (UID: "44a7b927-3900-4a1f-9270-b72dd3239865"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:04:00.739609 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.739584    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-dshm" pod "44a7b927-3900-4a1f-9270-b72dd3239865" (UID: "44a7b927-3900-4a1f-9270-b72dd3239865"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:00.740594 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.740574    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tls-certs" pod "2ba930d7-6e52-45ac-a9c5-95e03546ee2d" (UID: "2ba930d7-6e52-45ac-a9c5-95e03546ee2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:04:00.752501 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.752457    5647 generic.go:356] "Generic (PLEG): container finished" podID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerID="f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d" exitCode=0
Nov 03 23:04:00.752629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.752506    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"
Nov 03 23:04:00.752629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.752597    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" event={"ID":"2ba930d7-6e52-45ac-a9c5-95e03546ee2d","Type":"ContainerDied","Data":"f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d"}
Nov 03 23:04:00.752783 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.752635    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh" event={"ID":"2ba930d7-6e52-45ac-a9c5-95e03546ee2d","Type":"ContainerDied","Data":"03658dc7abf79ffd95aa762e28caec44383bb9afd32b3f7c279b27fda60e80f4"}
Nov 03 23:04:00.752783 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.752657    5647 scope.go:122] "RemoveContainer" containerID="f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d"
Nov 03 23:04:00.754876 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.754849    5647 generic.go:356] "Generic (PLEG): container finished" podID="44a7b927-3900-4a1f-9270-b72dd3239865" containerID="b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7" exitCode=0
Nov 03 23:04:00.755060 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.755043    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" event={"ID":"44a7b927-3900-4a1f-9270-b72dd3239865","Type":"ContainerDied","Data":"b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7"}
Nov 03 23:04:00.755146 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.755077    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277" event={"ID":"44a7b927-3900-4a1f-9270-b72dd3239865","Type":"ContainerDied","Data":"8d3e27382118d1956cb8a8da9462fa6b92644c6521308a40874f124b58a38d05"}
Nov 03 23:04:00.755146 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.755135    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"
Nov 03 23:04:00.771116 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.771096    5647 scope.go:122] "RemoveContainer" containerID="f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d"
Nov 03 23:04:00.771548 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:00.771523    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d\": container with ID starting with f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d not found: ID does not exist" containerID="f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d"
Nov 03 23:04:00.771715 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.771554    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d"} err="failed to get container status \"f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d\": rpc error: code = NotFound desc = could not find container \"f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d\": container with ID starting with f36cbf27a938622afdd4c38b616eff9599771219669a174a170de93d806f570d not found: ID does not exist"
Nov 03 23:04:00.771715 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.771574    5647 scope.go:122] "RemoveContainer" containerID="b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7"
Nov 03 23:04:00.781196 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.781176    5647 scope.go:122] "RemoveContainer" containerID="b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7"
Nov 03 23:04:00.781464 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:00.781445    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7\": container with ID starting with b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7 not found: ID does not exist" containerID="b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7"
Nov 03 23:04:00.781506 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.781473    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7"} err="failed to get container status \"b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7\": rpc error: code = NotFound desc = could not find container \"b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7\": container with ID starting with b04ffe6bf34e70dc24aeb29903e36fcecc97758906107f41183da9eec3b536b7 not found: ID does not exist"
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837445    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837475    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837488    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw49c\" (UniqueName: \"kubernetes.io/projected/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-kube-api-access-hw49c\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837503    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7nlz\" (UniqueName: \"kubernetes.io/projected/44a7b927-3900-4a1f-9270-b72dd3239865-kube-api-access-k7nlz\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837523    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837536    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837547    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ba930d7-6e52-45ac-a9c5-95e03546ee2d-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837568    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a7b927-3900-4a1f-9270-b72dd3239865-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.837612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.837583    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a7b927-3900-4a1f-9270-b72dd3239865-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:00.941660 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.941629    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"]
Nov 03 23:04:00.944266 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:00.944246    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/distinct-2-w0-b64e2e-kserve-7459659c68-x2277"]
Nov 03 23:04:01.050634 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:01.050588    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"]
Nov 03 23:04:01.056222 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:01.056192    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b64e2e/premium-w0-b64e2e-kserve-779c7f9d7b-tmpbh"]
Nov 03 23:04:02.697004 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:02.696970    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" path="/var/lib/kubelet/pods/2ba930d7-6e52-45ac-a9c5-95e03546ee2d/volumes"
Nov 03 23:04:02.697442 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:02.697427    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" path="/var/lib/kubelet/pods/44a7b927-3900-4a1f-9270-b72dd3239865/volumes"
Nov 03 23:04:16.043188 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:16.043146    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:04:19.690898 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:19.690858    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"]
Nov 03 23:04:19.745011 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:19.744977    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"]
Nov 03 23:04:24.691164 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:24.691129    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6" podUID="0417d2a7-b9ce-484c-89af-c7a80319b260" containerName="payload-processing" containerID="cri-o://a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458" gracePeriod=25
Nov 03 23:04:24.745929 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:24.745872    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7" podUID="b834e991-6ce5-4d77-84ca-b8f3a2225a84" containerName="payload-pre-processing" containerID="cri-o://5c9bd9d55bc5dbac44325565a93f80b70b1254bc901db4447811162de5c9a7c7" gracePeriod=25
Nov 03 23:04:25.027869 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.027840    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 23:04:25.039110 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.039074    5647 generic.go:356] "Generic (PLEG): container finished" podID="0417d2a7-b9ce-484c-89af-c7a80319b260" containerID="a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458" exitCode=0
Nov 03 23:04:25.039110 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.039119    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"
Nov 03 23:04:25.039324 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.039157    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6" event={"ID":"0417d2a7-b9ce-484c-89af-c7a80319b260","Type":"ContainerDied","Data":"a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458"}
Nov 03 23:04:25.039324 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.039223    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6" event={"ID":"0417d2a7-b9ce-484c-89af-c7a80319b260","Type":"ContainerDied","Data":"4fb7087b776813d696783e65629ba3fec9d283d2538d54f33fdac6b91d8c4552"}
Nov 03 23:04:25.039324 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.039253    5647 scope.go:122] "RemoveContainer" containerID="a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458"
Nov 03 23:04:25.040948 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.040912    5647 generic.go:356] "Generic (PLEG): container finished" podID="b834e991-6ce5-4d77-84ca-b8f3a2225a84" containerID="5c9bd9d55bc5dbac44325565a93f80b70b1254bc901db4447811162de5c9a7c7" exitCode=0
Nov 03 23:04:25.041102 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.040989    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7" event={"ID":"b834e991-6ce5-4d77-84ca-b8f3a2225a84","Type":"ContainerDied","Data":"5c9bd9d55bc5dbac44325565a93f80b70b1254bc901db4447811162de5c9a7c7"}
Nov 03 23:04:25.050339 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.050324    5647 scope.go:122] "RemoveContainer" containerID="a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458"
Nov 03 23:04:25.050619 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:25.050602    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458\": container with ID starting with a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458 not found: ID does not exist" containerID="a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458"
Nov 03 23:04:25.050706 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.050629    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458"} err="failed to get container status \"a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458\": rpc error: code = NotFound desc = could not find container \"a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458\": container with ID starting with a62dec009767298eb7706eaf386d10f0fce53dfeb00871b9f9d9f9725c1b3458 not found: ID does not exist"
Nov 03 23:04:25.053428 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.053408    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 23:04:25.159508 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.159476    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0417d2a7-b9ce-484c-89af-c7a80319b260-kube-api-access-lgt4t\" (UniqueName: \"kubernetes.io/projected/0417d2a7-b9ce-484c-89af-c7a80319b260-kube-api-access-lgt4t\") pod \"0417d2a7-b9ce-484c-89af-c7a80319b260\" (UID: \"0417d2a7-b9ce-484c-89af-c7a80319b260\") "
Nov 03 23:04:25.159718 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.159609    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b834e991-6ce5-4d77-84ca-b8f3a2225a84-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b834e991-6ce5-4d77-84ca-b8f3a2225a84-plugins-config-volume\") pod \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\" (UID: \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\") "
Nov 03 23:04:25.159718 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.159637    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b834e991-6ce5-4d77-84ca-b8f3a2225a84-kube-api-access-dmb76\" (UniqueName: \"kubernetes.io/projected/b834e991-6ce5-4d77-84ca-b8f3a2225a84-kube-api-access-dmb76\") pod \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\" (UID: \"b834e991-6ce5-4d77-84ca-b8f3a2225a84\") "
Nov 03 23:04:25.159954 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.159793    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0417d2a7-b9ce-484c-89af-c7a80319b260-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0417d2a7-b9ce-484c-89af-c7a80319b260-plugins-config-volume\") pod \"0417d2a7-b9ce-484c-89af-c7a80319b260\" (UID: \"0417d2a7-b9ce-484c-89af-c7a80319b260\") "
Nov 03 23:04:25.160119 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.160092    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b834e991-6ce5-4d77-84ca-b8f3a2225a84-plugins-config-volume" pod "b834e991-6ce5-4d77-84ca-b8f3a2225a84" (UID: "b834e991-6ce5-4d77-84ca-b8f3a2225a84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:04:25.160197 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.160127    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0417d2a7-b9ce-484c-89af-c7a80319b260-plugins-config-volume" pod "0417d2a7-b9ce-484c-89af-c7a80319b260" (UID: "0417d2a7-b9ce-484c-89af-c7a80319b260"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:04:25.161370 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.160838    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b834e991-6ce5-4d77-84ca-b8f3a2225a84-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:25.161370 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.160868    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0417d2a7-b9ce-484c-89af-c7a80319b260-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:25.161997 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.161971    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0417d2a7-b9ce-484c-89af-c7a80319b260-kube-api-access-lgt4t" pod "0417d2a7-b9ce-484c-89af-c7a80319b260" (UID: "0417d2a7-b9ce-484c-89af-c7a80319b260"). InnerVolumeSpecName "kube-api-access-lgt4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:25.162102 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.162023    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b834e991-6ce5-4d77-84ca-b8f3a2225a84-kube-api-access-dmb76" pod "b834e991-6ce5-4d77-84ca-b8f3a2225a84" (UID: "b834e991-6ce5-4d77-84ca-b8f3a2225a84"). InnerVolumeSpecName "kube-api-access-dmb76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:25.262296 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.262215    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmb76\" (UniqueName: \"kubernetes.io/projected/b834e991-6ce5-4d77-84ca-b8f3a2225a84-kube-api-access-dmb76\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:25.262296 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.262244    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgt4t\" (UniqueName: \"kubernetes.io/projected/0417d2a7-b9ce-484c-89af-c7a80319b260-kube-api-access-lgt4t\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:25.461200 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.461166    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"]
Nov 03 23:04:25.464033 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:25.464008    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6"]
Nov 03 23:04:26.049932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:26.049843    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"
Nov 03 23:04:26.049932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:26.049867    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7" event={"ID":"b834e991-6ce5-4d77-84ca-b8f3a2225a84","Type":"ContainerDied","Data":"b040f66050ab1321fcf152505287dae00d2ca68c38f51daba6f8e223b510c3fa"}
Nov 03 23:04:26.049932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:26.049925    5647 scope.go:122] "RemoveContainer" containerID="5c9bd9d55bc5dbac44325565a93f80b70b1254bc901db4447811162de5c9a7c7"
Nov 03 23:04:26.214319 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:26.214287    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"]
Nov 03 23:04:26.264300 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:26.264259    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7"]
Nov 03 23:04:26.697001 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:26.696968    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0417d2a7-b9ce-484c-89af-c7a80319b260" path="/var/lib/kubelet/pods/0417d2a7-b9ce-484c-89af-c7a80319b260/volumes"
Nov 03 23:04:26.697362 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:26.697348    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b834e991-6ce5-4d77-84ca-b8f3a2225a84" path="/var/lib/kubelet/pods/b834e991-6ce5-4d77-84ca-b8f3a2225a84/volumes"
Nov 03 23:04:28.031285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:28.031247    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"]
Nov 03 23:04:28.031751 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:28.031472    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy" containerID="cri-o://30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07" gracePeriod=30
Nov 03 23:04:33.317017 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.316953    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 23:04:33.375185 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375151    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83443efc-b020-45fb-a722-0e7eb4e5c719-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83443efc-b020-45fb-a722-0e7eb4e5c719-istiod-ca-cert\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375354 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375194    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-token\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-token\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375354 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375225    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-kube-api-access-w4wmd\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-kube-api-access-w4wmd\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375354 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375273    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-envoy\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375354 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375298    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-socket\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375585 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375395    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-credential-socket\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375585 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375472    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-podinfo\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375585 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375511    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-data\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375585 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375565    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-certs\") pod \"83443efc-b020-45fb-a722-0e7eb4e5c719\" (UID: \"83443efc-b020-45fb-a722-0e7eb4e5c719\") "
Nov 03 23:04:33.375836 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375714    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-socket" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:33.375836 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375727    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83443efc-b020-45fb-a722-0e7eb4e5c719-istiod-ca-cert" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:04:33.375951 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.375905    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-data" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:33.376091 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.376035    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-credential-socket" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:33.376265 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.376055    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-certs" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:33.376323 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.376308    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83443efc-b020-45fb-a722-0e7eb4e5c719-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:33.376389 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.376327    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:33.376389 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.376345    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:33.376389 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.376358    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:33.376389 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.376370    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:33.377768 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.377738    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-envoy" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:33.377880 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.377788    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-token" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:33.377880 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.377801    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-kube-api-access-w4wmd" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "kube-api-access-w4wmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:33.377955 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.377925    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-podinfo" pod "83443efc-b020-45fb-a722-0e7eb4e5c719" (UID: "83443efc-b020-45fb-a722-0e7eb4e5c719"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:04:33.477782 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.477743    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:33.477782 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.477779    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:33.477782 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.477789    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4wmd\" (UniqueName: \"kubernetes.io/projected/83443efc-b020-45fb-a722-0e7eb4e5c719-kube-api-access-w4wmd\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:33.478036 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:33.477800    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83443efc-b020-45fb-a722-0e7eb4e5c719-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:34.107208 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.107173    5647 generic.go:356] "Generic (PLEG): container finished" podID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerID="30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07" exitCode=0
Nov 03 23:04:34.107406 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.107218    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"
Nov 03 23:04:34.107406 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.107259    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" event={"ID":"83443efc-b020-45fb-a722-0e7eb4e5c719","Type":"ContainerDied","Data":"30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07"}
Nov 03 23:04:34.107406 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.107312    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4" event={"ID":"83443efc-b020-45fb-a722-0e7eb4e5c719","Type":"ContainerDied","Data":"71ee101a4f9642a97dea194f83985d22247e3f17d9a28da5975111c0997d7cb8"}
Nov 03 23:04:34.107406 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.107335    5647 scope.go:122] "RemoveContainer" containerID="30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07"
Nov 03 23:04:34.119314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.119297    5647 scope.go:122] "RemoveContainer" containerID="30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07"
Nov 03 23:04:34.119594 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:34.119574    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07\": container with ID starting with 30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07 not found: ID does not exist" containerID="30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07"
Nov 03 23:04:34.119649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.119602    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07"} err="failed to get container status \"30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07\": rpc error: code = NotFound desc = could not find container \"30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07\": container with ID starting with 30372b83ae88708955276120e5b03eda0d486c7bad2121cb25d2f8c84317af07 not found: ID does not exist"
Nov 03 23:04:34.322477 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.322441    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"]
Nov 03 23:04:34.372286 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.372212    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4"]
Nov 03 23:04:34.697862 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:34.697782    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" path="/var/lib/kubelet/pods/83443efc-b020-45fb-a722-0e7eb4e5c719/volumes"
Nov 03 23:04:38.677086 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:38.677052    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"]
Nov 03 23:04:38.782087 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:38.782053    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"]
Nov 03 23:04:43.678017 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:43.677956    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp" podUID="3ca465ff-12ce-41df-8642-9b0cd07dc68b" containerName="payload-processing" containerID="cri-o://7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473" gracePeriod=25
Nov 03 23:04:43.783232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:43.783186    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7" podUID="bc98e8b3-e780-4bc2-bc7f-1db2f505becc" containerName="payload-pre-processing" containerID="cri-o://40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7" gracePeriod=25
Nov 03 23:04:43.963215 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:43.963181    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:04:44.049332 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.049302    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ca465ff-12ce-41df-8642-9b0cd07dc68b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ca465ff-12ce-41df-8642-9b0cd07dc68b-plugins-config-volume\") pod \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\" (UID: \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\") "
Nov 03 23:04:44.049534 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.049463    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ca465ff-12ce-41df-8642-9b0cd07dc68b-kube-api-access-rsrm5\" (UniqueName: \"kubernetes.io/projected/3ca465ff-12ce-41df-8642-9b0cd07dc68b-kube-api-access-rsrm5\") pod \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\" (UID: \"3ca465ff-12ce-41df-8642-9b0cd07dc68b\") "
Nov 03 23:04:44.049711 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.049659    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ca465ff-12ce-41df-8642-9b0cd07dc68b-plugins-config-volume" pod "3ca465ff-12ce-41df-8642-9b0cd07dc68b" (UID: "3ca465ff-12ce-41df-8642-9b0cd07dc68b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:04:44.049964 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.049948    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ca465ff-12ce-41df-8642-9b0cd07dc68b-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:44.051900 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.051871    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ca465ff-12ce-41df-8642-9b0cd07dc68b-kube-api-access-rsrm5" pod "3ca465ff-12ce-41df-8642-9b0cd07dc68b" (UID: "3ca465ff-12ce-41df-8642-9b0cd07dc68b"). InnerVolumeSpecName "kube-api-access-rsrm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:44.075646 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.075617    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:04:44.150580 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.150551    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-plugins-config-volume\") pod \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\" (UID: \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\") "
Nov 03 23:04:44.150821 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.150763    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-kube-api-access-h5bpb\" (UniqueName: \"kubernetes.io/projected/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-kube-api-access-h5bpb\") pod \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\" (UID: \"bc98e8b3-e780-4bc2-bc7f-1db2f505becc\") "
Nov 03 23:04:44.150981 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.150961    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-plugins-config-volume" pod "bc98e8b3-e780-4bc2-bc7f-1db2f505becc" (UID: "bc98e8b3-e780-4bc2-bc7f-1db2f505becc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:04:44.151257 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.151239    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:44.151331 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.151263    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsrm5\" (UniqueName: \"kubernetes.io/projected/3ca465ff-12ce-41df-8642-9b0cd07dc68b-kube-api-access-rsrm5\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:44.153107 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.153086    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-kube-api-access-h5bpb" pod "bc98e8b3-e780-4bc2-bc7f-1db2f505becc" (UID: "bc98e8b3-e780-4bc2-bc7f-1db2f505becc"). InnerVolumeSpecName "kube-api-access-h5bpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:44.193838 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.193762    5647 generic.go:356] "Generic (PLEG): container finished" podID="3ca465ff-12ce-41df-8642-9b0cd07dc68b" containerID="7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473" exitCode=0
Nov 03 23:04:44.193838 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.193806    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"
Nov 03 23:04:44.194013 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.193844    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp" event={"ID":"3ca465ff-12ce-41df-8642-9b0cd07dc68b","Type":"ContainerDied","Data":"7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473"}
Nov 03 23:04:44.194013 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.193900    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp" event={"ID":"3ca465ff-12ce-41df-8642-9b0cd07dc68b","Type":"ContainerDied","Data":"93fb9b6f4d1aa8a3bd95984f14241bced32204c2dc4f2fa1c7600f414b3e677e"}
Nov 03 23:04:44.194013 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.193922    5647 scope.go:122] "RemoveContainer" containerID="7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473"
Nov 03 23:04:44.195390 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.195371    5647 generic.go:356] "Generic (PLEG): container finished" podID="bc98e8b3-e780-4bc2-bc7f-1db2f505becc" containerID="40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7" exitCode=0
Nov 03 23:04:44.195498 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.195414    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"
Nov 03 23:04:44.195498 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.195450    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7" event={"ID":"bc98e8b3-e780-4bc2-bc7f-1db2f505becc","Type":"ContainerDied","Data":"40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7"}
Nov 03 23:04:44.195603 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.195498    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7" event={"ID":"bc98e8b3-e780-4bc2-bc7f-1db2f505becc","Type":"ContainerDied","Data":"c82d7606293354263dd43d187134f9e5c7f76e26c39efedd9457c982974ba147"}
Nov 03 23:04:44.206130 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.206111    5647 scope.go:122] "RemoveContainer" containerID="7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473"
Nov 03 23:04:44.206694 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:44.206656    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473\": container with ID starting with 7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473 not found: ID does not exist" containerID="7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473"
Nov 03 23:04:44.206797 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.206700    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473"} err="failed to get container status \"7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473\": rpc error: code = NotFound desc = could not find container \"7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473\": container with ID starting with 7dfdf81b7267077801197318e4dd07371ad905f75ff4766d20d850bb1c59e473 not found: ID does not exist"
Nov 03 23:04:44.206797 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.206716    5647 scope.go:122] "RemoveContainer" containerID="40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7"
Nov 03 23:04:44.216686 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.216651    5647 scope.go:122] "RemoveContainer" containerID="40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7"
Nov 03 23:04:44.216974 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:44.216956    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7\": container with ID starting with 40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7 not found: ID does not exist" containerID="40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7"
Nov 03 23:04:44.217050 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.216980    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7"} err="failed to get container status \"40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7\": rpc error: code = NotFound desc = could not find container \"40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7\": container with ID starting with 40a7b328a19c5132bd5c2bb5eb9468db8718e3cbf81901dd1169ff49e80af8d7 not found: ID does not exist"
Nov 03 23:04:44.253197 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.253168    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5bpb\" (UniqueName: \"kubernetes.io/projected/bc98e8b3-e780-4bc2-bc7f-1db2f505becc-kube-api-access-h5bpb\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:44.349046 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.349011    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"]
Nov 03 23:04:44.352582 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.352555    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp"]
Nov 03 23:04:44.460827 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.460797    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"]
Nov 03 23:04:44.466898 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.466871    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7"]
Nov 03 23:04:44.697274 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.697246    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ca465ff-12ce-41df-8642-9b0cd07dc68b" path="/var/lib/kubelet/pods/3ca465ff-12ce-41df-8642-9b0cd07dc68b/volumes"
Nov 03 23:04:44.697728 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:44.697713    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc98e8b3-e780-4bc2-bc7f-1db2f505becc" path="/var/lib/kubelet/pods/bc98e8b3-e780-4bc2-bc7f-1db2f505becc/volumes"
Nov 03 23:04:46.045838 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:46.045798    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:04:48.962452 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:48.962421    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"]
Nov 03 23:04:48.962986 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:48.962627    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy" containerID="cri-o://5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3" gracePeriod=30
Nov 03 23:04:54.230683 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.230600    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 23:04:54.240055 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240028    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-socket\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240193 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240091    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-kube-api-access-4cjbc\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-kube-api-access-4cjbc\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240193 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240110    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-data\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240193 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240155    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-istio-token\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-istio-token\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240193 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240173    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-envoy\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240419 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240273    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bdd92147-66a8-49de-9612-468a9c71cac1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdd92147-66a8-49de-9612-468a9c71cac1-istio-podinfo\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240419 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240333    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-certs\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240419 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240329    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-socket" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:54.240419 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240400    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-credential-socket\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240644 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240457    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdd92147-66a8-49de-9612-468a9c71cac1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdd92147-66a8-49de-9612-468a9c71cac1-istiod-ca-cert\") pod \"bdd92147-66a8-49de-9612-468a9c71cac1\" (UID: \"bdd92147-66a8-49de-9612-468a9c71cac1\") "
Nov 03 23:04:54.240644 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240457    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-data" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:54.240797 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240619    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-certs" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:54.240797 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240725    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-credential-socket" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:54.240979 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.240960    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdd92147-66a8-49de-9612-468a9c71cac1-istiod-ca-cert" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:04:54.241195 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.241176    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.241234 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.241203    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.241234 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.241221    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.241301 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.241236    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdd92147-66a8-49de-9612-468a9c71cac1-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.241301 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.241249    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.242484 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.242465    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-envoy" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:04:54.242610 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.242592    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-istio-token" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:54.242986 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.242965    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bdd92147-66a8-49de-9612-468a9c71cac1-istio-podinfo" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:04:54.243055 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.243005    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-kube-api-access-4cjbc" pod "bdd92147-66a8-49de-9612-468a9c71cac1" (UID: "bdd92147-66a8-49de-9612-468a9c71cac1"). InnerVolumeSpecName "kube-api-access-4cjbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:04:54.299162 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.299129    5647 generic.go:356] "Generic (PLEG): container finished" podID="bdd92147-66a8-49de-9612-468a9c71cac1" containerID="5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3" exitCode=0
Nov 03 23:04:54.299349 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.299188    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"
Nov 03 23:04:54.299349 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.299220    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" event={"ID":"bdd92147-66a8-49de-9612-468a9c71cac1","Type":"ContainerDied","Data":"5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3"}
Nov 03 23:04:54.299349 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.299323    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5" event={"ID":"bdd92147-66a8-49de-9612-468a9c71cac1","Type":"ContainerDied","Data":"d3048b3dfe441e8db8fba60731a81c8533b95d12527b9c46194c094365559a05"}
Nov 03 23:04:54.299499 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.299360    5647 scope.go:122] "RemoveContainer" containerID="5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3"
Nov 03 23:04:54.313204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.313183    5647 scope.go:122] "RemoveContainer" containerID="497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e"
Nov 03 23:04:54.322962 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.322940    5647 scope.go:122] "RemoveContainer" containerID="5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3"
Nov 03 23:04:54.323254 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:54.323233    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3\": container with ID starting with 5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3 not found: ID does not exist" containerID="5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3"
Nov 03 23:04:54.323323 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.323262    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3"} err="failed to get container status \"5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3\": rpc error: code = NotFound desc = could not find container \"5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3\": container with ID starting with 5a171d7715f8a586ba36b43b88c46508710da633dc00e89529146bb8171d23c3 not found: ID does not exist"
Nov 03 23:04:54.323323 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.323280    5647 scope.go:122] "RemoveContainer" containerID="497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e"
Nov 03 23:04:54.323555 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:04:54.323535    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e\": container with ID starting with 497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e not found: ID does not exist" containerID="497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e"
Nov 03 23:04:54.323607 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.323560    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e"} err="failed to get container status \"497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e\": rpc error: code = NotFound desc = could not find container \"497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e\": container with ID starting with 497a3c563815464c6a4a525d7d7cf7a17676ce5f681296bc0beaed5a6972369e not found: ID does not exist"
Nov 03 23:04:54.342422 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.342397    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdd92147-66a8-49de-9612-468a9c71cac1-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.342517 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.342425    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cjbc\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-kube-api-access-4cjbc\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.342517 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.342439    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdd92147-66a8-49de-9612-468a9c71cac1-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.342517 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.342452    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdd92147-66a8-49de-9612-468a9c71cac1-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:04:54.464455 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.464425    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"]
Nov 03 23:04:54.468570 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.468545    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5"]
Nov 03 23:04:54.697901 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.697821    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" path="/var/lib/kubelet/pods/bdd92147-66a8-49de-9612-468a9c71cac1/volumes"
Nov 03 23:04:54.859913 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.859883    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"]
Nov 03 23:04:54.860140 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:54.860116    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" containerID="cri-o://d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397" gracePeriod=30
Nov 03 23:04:55.938612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:55.938575    5647 patch_prober.go:29] interesting pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:04:55.939057 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:04:55.938644    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:05:00.129100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.129069    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 23:05:00.229632 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.229597    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-podinfo\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.229860 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.229706    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-certs\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.229860 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.229733    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-envoy\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.229860 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.229760    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-socket\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.229860 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.229855    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8dfa5234-0fb8-4b64-b263-313228dc237e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dfa5234-0fb8-4b64-b263-313228dc237e-istiod-ca-cert\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.230106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.229920    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-data\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.230106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.229985    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-token\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-token\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.230106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.230047    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-kube-api-access-gjmr9\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-kube-api-access-gjmr9\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.230106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.230067    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-certs" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:05:00.230106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.230089    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-socket" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:05:00.230106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.230102    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-credential-socket\") pod \"8dfa5234-0fb8-4b64-b263-313228dc237e\" (UID: \"8dfa5234-0fb8-4b64-b263-313228dc237e\") "
Nov 03 23:05:00.230467 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.230292    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-data" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:05:00.230467 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.230334    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dfa5234-0fb8-4b64-b263-313228dc237e-istiod-ca-cert" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:05:00.230619 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.230563    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-credential-socket" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:05:00.231053 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.231028    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.231207 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.231191    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.231310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.231286    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.231310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.231309    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dfa5234-0fb8-4b64-b263-313228dc237e-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.231834 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.231323    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.232491 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.232388    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-envoy" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:05:00.233155 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.233131    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-podinfo" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:05:00.233313 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.233293    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-kube-api-access-gjmr9" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "kube-api-access-gjmr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:05:00.233420 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.233405    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-token" pod "8dfa5234-0fb8-4b64-b263-313228dc237e" (UID: "8dfa5234-0fb8-4b64-b263-313228dc237e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:05:00.333077 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.332991    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.333077 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.333024    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjmr9\" (UniqueName: \"kubernetes.io/projected/8dfa5234-0fb8-4b64-b263-313228dc237e-kube-api-access-gjmr9\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.333077 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.333041    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.333077 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.333053    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dfa5234-0fb8-4b64-b263-313228dc237e-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:00.340036 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.340003    5647 generic.go:356] "Generic (PLEG): container finished" podID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerID="d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397" exitCode=0
Nov 03 23:05:00.340198 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.340049    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"
Nov 03 23:05:00.340198 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.340085    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" event={"ID":"8dfa5234-0fb8-4b64-b263-313228dc237e","Type":"ContainerDied","Data":"d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397"}
Nov 03 23:05:00.340198 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.340151    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs" event={"ID":"8dfa5234-0fb8-4b64-b263-313228dc237e","Type":"ContainerDied","Data":"6493a275d9bb6db4d80f29ddf7c0b6146393de8299489f08c578d257c8342d14"}
Nov 03 23:05:00.340198 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.340192    5647 scope.go:122] "RemoveContainer" containerID="d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397"
Nov 03 23:05:00.358706 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.358661    5647 scope.go:122] "RemoveContainer" containerID="d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397"
Nov 03 23:05:00.359059 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:05:00.359034    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397\": container with ID starting with d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397 not found: ID does not exist" containerID="d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397"
Nov 03 23:05:00.359173 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.359066    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397"} err="failed to get container status \"d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397\": rpc error: code = NotFound desc = could not find container \"d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397\": container with ID starting with d63cedf37527ebaa48905713ec80efc9456a5040ff104e255ede3decda67b397 not found: ID does not exist"
Nov 03 23:05:00.546983 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.546946    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"]
Nov 03 23:05:00.554744 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.554714    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs"]
Nov 03 23:05:00.697575 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:00.697498    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" path="/var/lib/kubelet/pods/8dfa5234-0fb8-4b64-b263-313228dc237e/volumes"
Nov 03 23:05:01.286942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:01.286911    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"]
Nov 03 23:05:06.287310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:06.287247    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5" podUID="4e4940fe-818b-4b7a-a64d-33f8a8c0e954" containerName="payload-processing" containerID="cri-o://d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801" gracePeriod=25
Nov 03 23:05:06.570038 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:06.570012    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:05:06.633475 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:06.633380    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-kube-api-access-kjd4f\" (UniqueName: \"kubernetes.io/projected/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-kube-api-access-kjd4f\") pod \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\" (UID: \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\") "
Nov 03 23:05:06.633736 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:06.633717    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-plugins-config-volume\") pod \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\" (UID: \"4e4940fe-818b-4b7a-a64d-33f8a8c0e954\") "
Nov 03 23:05:06.634043 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:06.634019    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-plugins-config-volume" pod "4e4940fe-818b-4b7a-a64d-33f8a8c0e954" (UID: "4e4940fe-818b-4b7a-a64d-33f8a8c0e954"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:05:06.634268 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:06.634254    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:06.635741 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:06.635717    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-kube-api-access-kjd4f" pod "4e4940fe-818b-4b7a-a64d-33f8a8c0e954" (UID: "4e4940fe-818b-4b7a-a64d-33f8a8c0e954"). InnerVolumeSpecName "kube-api-access-kjd4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:05:06.736031 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:06.735985    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjd4f\" (UniqueName: \"kubernetes.io/projected/4e4940fe-818b-4b7a-a64d-33f8a8c0e954-kube-api-access-kjd4f\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:07.393951 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.393911    5647 generic.go:356] "Generic (PLEG): container finished" podID="4e4940fe-818b-4b7a-a64d-33f8a8c0e954" containerID="d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801" exitCode=0
Nov 03 23:05:07.394459 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.393970    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"
Nov 03 23:05:07.394459 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.393988    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5" event={"ID":"4e4940fe-818b-4b7a-a64d-33f8a8c0e954","Type":"ContainerDied","Data":"d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801"}
Nov 03 23:05:07.394459 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.394083    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5" event={"ID":"4e4940fe-818b-4b7a-a64d-33f8a8c0e954","Type":"ContainerDied","Data":"377c0f7e753afbded65304d914eb22a0d948827a7705257faeb066a31d815f21"}
Nov 03 23:05:07.394459 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.394117    5647 scope.go:122] "RemoveContainer" containerID="d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801"
Nov 03 23:05:07.404146 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.404123    5647 scope.go:122] "RemoveContainer" containerID="d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801"
Nov 03 23:05:07.404448 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:05:07.404425    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801\": container with ID starting with d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801 not found: ID does not exist" containerID="d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801"
Nov 03 23:05:07.404497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.404456    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801"} err="failed to get container status \"d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801\": rpc error: code = NotFound desc = could not find container \"d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801\": container with ID starting with d10342e571c6f468fdc2b23a1c84a06784190e687fc80fd3da168bec72811801 not found: ID does not exist"
Nov 03 23:05:07.559314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.559280    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"]
Nov 03 23:05:07.565125 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:07.565098    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5"]
Nov 03 23:05:08.696823 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:08.696790    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e4940fe-818b-4b7a-a64d-33f8a8c0e954" path="/var/lib/kubelet/pods/4e4940fe-818b-4b7a-a64d-33f8a8c0e954/volumes"
Nov 03 23:05:16.043805 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:05:16.043757    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:05:16.750528 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:16.750484    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:05:32.281335 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.281295    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"]
Nov 03 23:05:32.281939 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.281499    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b" podUID="4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" containerName="maas-api" containerID="cri-o://5c42ff4696521824f792d99bffb1ba0ba17d1c3fbc081a13918b624e223ce555" gracePeriod=30
Nov 03 23:05:32.586368 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.586335    5647 generic.go:356] "Generic (PLEG): container finished" podID="4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" containerID="5c42ff4696521824f792d99bffb1ba0ba17d1c3fbc081a13918b624e223ce555" exitCode=0
Nov 03 23:05:32.586491 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.586393    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b" event={"ID":"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863","Type":"ContainerDied","Data":"5c42ff4696521824f792d99bffb1ba0ba17d1c3fbc081a13918b624e223ce555"}
Nov 03 23:05:32.593544 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.593523    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 23:05:32.729425 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.729395    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-metrics-certs\") pod \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") "
Nov 03 23:05:32.729643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.729628    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-openshift-service-ca\") pod \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") "
Nov 03 23:05:32.729741 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.729708    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-kube-api-access-r9x44\" (UniqueName: \"kubernetes.io/projected/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-kube-api-access-r9x44\") pod \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") "
Nov 03 23:05:32.729807 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.729763    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls\") pod \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\" (UID: \"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863\") "
Nov 03 23:05:32.730038 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.730009    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-openshift-service-ca" pod "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" (UID: "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:05:32.730314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.730288    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:32.732285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.732261    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-metrics-certs" pod "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" (UID: "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:05:32.734990 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.734969    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls" pod "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" (UID: "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:05:32.736433 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.736411    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-kube-api-access-r9x44" pod "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" (UID: "4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863"). InnerVolumeSpecName "kube-api-access-r9x44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:05:32.852631 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.849286    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9x44\" (UniqueName: \"kubernetes.io/projected/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-kube-api-access-r9x44\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:32.852631 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.849322    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:32.852631 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:32.849338    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:33.593464 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:33.593435    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"
Nov 03 23:05:33.593908 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:33.593478    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b" event={"ID":"4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863","Type":"ContainerDied","Data":"b70e775b8c0dcb5f0bf37bdde3c3124692a2d94d6489d49a8a25149d97c7e391"}
Nov 03 23:05:33.593908 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:33.593536    5647 scope.go:122] "RemoveContainer" containerID="5c42ff4696521824f792d99bffb1ba0ba17d1c3fbc081a13918b624e223ce555"
Nov 03 23:05:33.836886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:33.836719    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"]
Nov 03 23:05:33.856790 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:33.854033    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b64e2e-76cb697d59-8fk9b"]
Nov 03 23:05:34.696442 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:34.696411    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" path="/var/lib/kubelet/pods/4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863/volumes"
Nov 03 23:05:36.017301 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:36.017272    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"]
Nov 03 23:05:36.133540 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:36.133493    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"]
Nov 03 23:05:41.018466 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.018422    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4" podUID="12830a1b-7618-41cf-860d-932a2ddd38a1" containerName="payload-processing" containerID="cri-o://a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc" gracePeriod=25
Nov 03 23:05:41.134179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.134142    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd" podUID="987eea5f-e434-4274-99fe-b0213675ef77" containerName="payload-pre-processing" containerID="cri-o://39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a" gracePeriod=25
Nov 03 23:05:41.322413 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.322385    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 23:05:41.444709 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.441308    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12830a1b-7618-41cf-860d-932a2ddd38a1-kube-api-access-tqpxn\" (UniqueName: \"kubernetes.io/projected/12830a1b-7618-41cf-860d-932a2ddd38a1-kube-api-access-tqpxn\") pod \"12830a1b-7618-41cf-860d-932a2ddd38a1\" (UID: \"12830a1b-7618-41cf-860d-932a2ddd38a1\") "
Nov 03 23:05:41.444709 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.441376    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12830a1b-7618-41cf-860d-932a2ddd38a1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12830a1b-7618-41cf-860d-932a2ddd38a1-plugins-config-volume\") pod \"12830a1b-7618-41cf-860d-932a2ddd38a1\" (UID: \"12830a1b-7618-41cf-860d-932a2ddd38a1\") "
Nov 03 23:05:41.455817 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.455727    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12830a1b-7618-41cf-860d-932a2ddd38a1-plugins-config-volume" pod "12830a1b-7618-41cf-860d-932a2ddd38a1" (UID: "12830a1b-7618-41cf-860d-932a2ddd38a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:05:41.476043 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.475977    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12830a1b-7618-41cf-860d-932a2ddd38a1-kube-api-access-tqpxn" pod "12830a1b-7618-41cf-860d-932a2ddd38a1" (UID: "12830a1b-7618-41cf-860d-932a2ddd38a1"). InnerVolumeSpecName "kube-api-access-tqpxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:05:41.477194 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.476910    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 23:05:41.553915 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.553835    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12830a1b-7618-41cf-860d-932a2ddd38a1-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:41.553915 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.553864    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqpxn\" (UniqueName: \"kubernetes.io/projected/12830a1b-7618-41cf-860d-932a2ddd38a1-kube-api-access-tqpxn\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:41.655171 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.655134    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/987eea5f-e434-4274-99fe-b0213675ef77-kube-api-access-grfq7\" (UniqueName: \"kubernetes.io/projected/987eea5f-e434-4274-99fe-b0213675ef77-kube-api-access-grfq7\") pod \"987eea5f-e434-4274-99fe-b0213675ef77\" (UID: \"987eea5f-e434-4274-99fe-b0213675ef77\") "
Nov 03 23:05:41.655343 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.655212    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/987eea5f-e434-4274-99fe-b0213675ef77-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/987eea5f-e434-4274-99fe-b0213675ef77-plugins-config-volume\") pod \"987eea5f-e434-4274-99fe-b0213675ef77\" (UID: \"987eea5f-e434-4274-99fe-b0213675ef77\") "
Nov 03 23:05:41.655597 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.655564    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/987eea5f-e434-4274-99fe-b0213675ef77-plugins-config-volume" pod "987eea5f-e434-4274-99fe-b0213675ef77" (UID: "987eea5f-e434-4274-99fe-b0213675ef77"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:05:41.655798 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.655782    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/987eea5f-e434-4274-99fe-b0213675ef77-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:41.657374 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.657353    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/987eea5f-e434-4274-99fe-b0213675ef77-kube-api-access-grfq7" pod "987eea5f-e434-4274-99fe-b0213675ef77" (UID: "987eea5f-e434-4274-99fe-b0213675ef77"). InnerVolumeSpecName "kube-api-access-grfq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:05:41.682047 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.682017    5647 generic.go:356] "Generic (PLEG): container finished" podID="987eea5f-e434-4274-99fe-b0213675ef77" containerID="39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a" exitCode=0
Nov 03 23:05:41.682203 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.682060    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"
Nov 03 23:05:41.682203 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.682107    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd" event={"ID":"987eea5f-e434-4274-99fe-b0213675ef77","Type":"ContainerDied","Data":"39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a"}
Nov 03 23:05:41.682203 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.682168    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd" event={"ID":"987eea5f-e434-4274-99fe-b0213675ef77","Type":"ContainerDied","Data":"79aef16e8538a9c2ee864dc930522f71c8d48703182f85acfb11ac4489bcf18a"}
Nov 03 23:05:41.682203 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.682197    5647 scope.go:122] "RemoveContainer" containerID="39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a"
Nov 03 23:05:41.684107 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.684080    5647 generic.go:356] "Generic (PLEG): container finished" podID="12830a1b-7618-41cf-860d-932a2ddd38a1" containerID="a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc" exitCode=0
Nov 03 23:05:41.684227 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.684116    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"
Nov 03 23:05:41.684227 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.684169    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4" event={"ID":"12830a1b-7618-41cf-860d-932a2ddd38a1","Type":"ContainerDied","Data":"a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc"}
Nov 03 23:05:41.684227 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.684223    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4" event={"ID":"12830a1b-7618-41cf-860d-932a2ddd38a1","Type":"ContainerDied","Data":"a071cd1efb389efeac926e38973128e43b46d1c583de231d8ae125ed71fd3572"}
Nov 03 23:05:41.693106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.693078    5647 scope.go:122] "RemoveContainer" containerID="39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a"
Nov 03 23:05:41.693383 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:05:41.693361    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a\": container with ID starting with 39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a not found: ID does not exist" containerID="39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a"
Nov 03 23:05:41.693467 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.693388    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a"} err="failed to get container status \"39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a\": rpc error: code = NotFound desc = could not find container \"39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a\": container with ID starting with 39dbbd0774268aca884e2bac0a84039e363070726a036af359db6331037d885a not found: ID does not exist"
Nov 03 23:05:41.693467 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.693405    5647 scope.go:122] "RemoveContainer" containerID="a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc"
Nov 03 23:05:41.702193 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.702177    5647 scope.go:122] "RemoveContainer" containerID="a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc"
Nov 03 23:05:41.702445 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:05:41.702429    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc\": container with ID starting with a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc not found: ID does not exist" containerID="a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc"
Nov 03 23:05:41.702501 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.702452    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc"} err="failed to get container status \"a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc\": rpc error: code = NotFound desc = could not find container \"a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc\": container with ID starting with a5c69e934db28e4597be8f4ea5d8e0c0fa46fef3384f0a5bd791f5c8b08152bc not found: ID does not exist"
Nov 03 23:05:41.756599 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.756569    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grfq7\" (UniqueName: \"kubernetes.io/projected/987eea5f-e434-4274-99fe-b0213675ef77-kube-api-access-grfq7\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:05:41.845134 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.845035    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"]
Nov 03 23:05:41.850316 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.850283    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd"]
Nov 03 23:05:41.962748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.962709    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"]
Nov 03 23:05:41.967645 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:41.967604    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4"]
Nov 03 23:05:42.697250 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:42.697204    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12830a1b-7618-41cf-860d-932a2ddd38a1" path="/var/lib/kubelet/pods/12830a1b-7618-41cf-860d-932a2ddd38a1/volumes"
Nov 03 23:05:42.697849 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:42.697828    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="987eea5f-e434-4274-99fe-b0213675ef77" path="/var/lib/kubelet/pods/987eea5f-e434-4274-99fe-b0213675ef77/volumes"
Nov 03 23:05:46.043128 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:05:46.043093    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:05:52.059723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:52.059690    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:05:53.252461 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:53.252427    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"]
Nov 03 23:05:53.301731 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:53.301655    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"]
Nov 03 23:05:58.931030 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:58.930985    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"]
Nov 03 23:05:58.931439 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:05:58.931181    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy" containerID="cri-o://b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9" gracePeriod=30
Nov 03 23:06:04.227155 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.226050    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.294761    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-envoy\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.294822    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-data\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.294876    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9406813d-528c-4d15-8276-7d13f292cb13-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9406813d-528c-4d15-8276-7d13f292cb13-istiod-ca-cert\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.294968    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-istio-token\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-istio-token\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.295023    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9406813d-528c-4d15-8276-7d13f292cb13-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9406813d-528c-4d15-8276-7d13f292cb13-istio-podinfo\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.295053    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-kube-api-access-46psr\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-kube-api-access-46psr\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.295124    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-socket\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.295207    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-credential-socket\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.295257    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-certs\") pod \"9406813d-528c-4d15-8276-7d13f292cb13\" (UID: \"9406813d-528c-4d15-8276-7d13f292cb13\") "
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.296802    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-certs" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.297201    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-socket" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.297411    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-credential-socket" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.300526    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-kube-api-access-46psr" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "kube-api-access-46psr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.300707    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-data" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.301021    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9406813d-528c-4d15-8276-7d13f292cb13-istio-podinfo" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.301244    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9406813d-528c-4d15-8276-7d13f292cb13-istiod-ca-cert" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.303375    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-envoy" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:04.303907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.303775    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-istio-token" pod "9406813d-528c-4d15-8276-7d13f292cb13" (UID: "9406813d-528c-4d15-8276-7d13f292cb13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:04.396563 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396528    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9406813d-528c-4d15-8276-7d13f292cb13-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.396563 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396559    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46psr\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-kube-api-access-46psr\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.396563 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396571    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.396833 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396580    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.396833 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396589    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.396833 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396598    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.396833 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396606    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9406813d-528c-4d15-8276-7d13f292cb13-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.396833 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396615    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9406813d-528c-4d15-8276-7d13f292cb13-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.396833 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.396623    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9406813d-528c-4d15-8276-7d13f292cb13-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:04.879284 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.879249    5647 generic.go:356] "Generic (PLEG): container finished" podID="9406813d-528c-4d15-8276-7d13f292cb13" containerID="b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9" exitCode=0
Nov 03 23:06:04.879468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.879294    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"
Nov 03 23:06:04.879468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.879337    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" event={"ID":"9406813d-528c-4d15-8276-7d13f292cb13","Type":"ContainerDied","Data":"b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9"}
Nov 03 23:06:04.879468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.879396    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp" event={"ID":"9406813d-528c-4d15-8276-7d13f292cb13","Type":"ContainerDied","Data":"99af68fa87886f505996636343dd62a87a9ea88fb62c4fbdfcfdda86120a29d7"}
Nov 03 23:06:04.879468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.879425    5647 scope.go:122] "RemoveContainer" containerID="b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9"
Nov 03 23:06:04.889132 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.889111    5647 scope.go:122] "RemoveContainer" containerID="b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9"
Nov 03 23:06:04.889417 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:04.889392    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9\": container with ID starting with b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9 not found: ID does not exist" containerID="b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9"
Nov 03 23:06:04.889522 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:04.889427    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9"} err="failed to get container status \"b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9\": rpc error: code = NotFound desc = could not find container \"b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9\": container with ID starting with b36b9f8ff1696377d7f4db49886be0fc1341c2d92d4d8ddc375699842cd143b9 not found: ID does not exist"
Nov 03 23:06:05.095829 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:05.095744    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"]
Nov 03 23:06:05.145012 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:05.144979    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp"]
Nov 03 23:06:06.695991 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:06.695955    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9406813d-528c-4d15-8276-7d13f292cb13" path="/var/lib/kubelet/pods/9406813d-528c-4d15-8276-7d13f292cb13/volumes"
Nov 03 23:06:08.273902 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.273864    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main" containerID="cri-o://cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad" gracePeriod=45
Nov 03 23:06:08.322882 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.322852    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main" containerID="cri-o://f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57" gracePeriod=45
Nov 03 23:06:08.591007 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.590976    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 23:06:08.616392 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.616371    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 23:06:08.649139 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649113    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tls-certs\" (UniqueName: \"kubernetes.io/secret/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tls-certs\") pod \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") "
Nov 03 23:06:08.649254 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649150    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-home\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-home\") pod \"28f92588-8375-4ef4-ae91-3ff783e71def\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") "
Nov 03 23:06:08.649322 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649271    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-home\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-home\") pod \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") "
Nov 03 23:06:08.649369 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649345    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-dshm\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-dshm\") pod \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") "
Nov 03 23:06:08.649468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649451    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-model-cache\") pod \"28f92588-8375-4ef4-ae91-3ff783e71def\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") "
Nov 03 23:06:08.649538 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649495    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/445d4bb2-6baf-44ea-887d-4d0443b9fe01-kube-api-access-6kknv\" (UniqueName: \"kubernetes.io/projected/445d4bb2-6baf-44ea-887d-4d0443b9fe01-kube-api-access-6kknv\") pod \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") "
Nov 03 23:06:08.649538 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649529    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-model-cache\") pod \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") "
Nov 03 23:06:08.649650 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649551    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-home" pod "445d4bb2-6baf-44ea-887d-4d0443b9fe01" (UID: "445d4bb2-6baf-44ea-887d-4d0443b9fe01"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:08.649650 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649579    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-dshm\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-dshm\") pod \"28f92588-8375-4ef4-ae91-3ff783e71def\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") "
Nov 03 23:06:08.649650 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649607    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-tmp-dir\") pod \"28f92588-8375-4ef4-ae91-3ff783e71def\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") "
Nov 03 23:06:08.649650 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649640    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28f92588-8375-4ef4-ae91-3ff783e71def-tls-certs\" (UniqueName: \"kubernetes.io/secret/28f92588-8375-4ef4-ae91-3ff783e71def-tls-certs\") pod \"28f92588-8375-4ef4-ae91-3ff783e71def\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") "
Nov 03 23:06:08.649913 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649660    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-model-cache" pod "28f92588-8375-4ef4-ae91-3ff783e71def" (UID: "28f92588-8375-4ef4-ae91-3ff783e71def"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:08.649913 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649715    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tmp-dir\") pod \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\" (UID: \"445d4bb2-6baf-44ea-887d-4d0443b9fe01\") "
Nov 03 23:06:08.649913 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649756    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28f92588-8375-4ef4-ae91-3ff783e71def-kube-api-access-sfvbn\" (UniqueName: \"kubernetes.io/projected/28f92588-8375-4ef4-ae91-3ff783e71def-kube-api-access-sfvbn\") pod \"28f92588-8375-4ef4-ae91-3ff783e71def\" (UID: \"28f92588-8375-4ef4-ae91-3ff783e71def\") "
Nov 03 23:06:08.649913 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649864    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-tmp-dir" pod "28f92588-8375-4ef4-ae91-3ff783e71def" (UID: "28f92588-8375-4ef4-ae91-3ff783e71def"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:08.650298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.649474    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-home" pod "28f92588-8375-4ef4-ae91-3ff783e71def" (UID: "28f92588-8375-4ef4-ae91-3ff783e71def"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:08.650298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.650237    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-model-cache" pod "445d4bb2-6baf-44ea-887d-4d0443b9fe01" (UID: "445d4bb2-6baf-44ea-887d-4d0443b9fe01"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:08.650452 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.650428    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tmp-dir" pod "445d4bb2-6baf-44ea-887d-4d0443b9fe01" (UID: "445d4bb2-6baf-44ea-887d-4d0443b9fe01"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:08.650815 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.650792    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.650932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.650818    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.650932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.650836    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.650932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.650850    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.650932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.650868    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.650932 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.650880    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.652129 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.652104    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/445d4bb2-6baf-44ea-887d-4d0443b9fe01-kube-api-access-6kknv" pod "445d4bb2-6baf-44ea-887d-4d0443b9fe01" (UID: "445d4bb2-6baf-44ea-887d-4d0443b9fe01"). InnerVolumeSpecName "kube-api-access-6kknv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:08.652129 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.652118    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28f92588-8375-4ef4-ae91-3ff783e71def-tls-certs" pod "28f92588-8375-4ef4-ae91-3ff783e71def" (UID: "28f92588-8375-4ef4-ae91-3ff783e71def"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:06:08.652253 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.652121    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tls-certs" pod "445d4bb2-6baf-44ea-887d-4d0443b9fe01" (UID: "445d4bb2-6baf-44ea-887d-4d0443b9fe01"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:06:08.652253 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.652166    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-dshm" pod "445d4bb2-6baf-44ea-887d-4d0443b9fe01" (UID: "445d4bb2-6baf-44ea-887d-4d0443b9fe01"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:08.652419 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.652401    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f92588-8375-4ef4-ae91-3ff783e71def-kube-api-access-sfvbn" pod "28f92588-8375-4ef4-ae91-3ff783e71def" (UID: "28f92588-8375-4ef4-ae91-3ff783e71def"). InnerVolumeSpecName "kube-api-access-sfvbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:08.652524 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.652507    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-dshm" pod "28f92588-8375-4ef4-ae91-3ff783e71def" (UID: "28f92588-8375-4ef4-ae91-3ff783e71def"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:08.752083 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.752042    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/445d4bb2-6baf-44ea-887d-4d0443b9fe01-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.752083 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.752078    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kknv\" (UniqueName: \"kubernetes.io/projected/445d4bb2-6baf-44ea-887d-4d0443b9fe01-kube-api-access-6kknv\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.752337 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.752094    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28f92588-8375-4ef4-ae91-3ff783e71def-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.752337 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.752109    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28f92588-8375-4ef4-ae91-3ff783e71def-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.752337 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.752122    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfvbn\" (UniqueName: \"kubernetes.io/projected/28f92588-8375-4ef4-ae91-3ff783e71def-kube-api-access-sfvbn\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.752337 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.752135    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/445d4bb2-6baf-44ea-887d-4d0443b9fe01-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:08.915021 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.914931    5647 generic.go:356] "Generic (PLEG): container finished" podID="28f92588-8375-4ef4-ae91-3ff783e71def" containerID="f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57" exitCode=0
Nov 03 23:06:08.915021 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.914982    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"
Nov 03 23:06:08.915021 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.915012    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" event={"ID":"28f92588-8375-4ef4-ae91-3ff783e71def","Type":"ContainerDied","Data":"f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57"}
Nov 03 23:06:08.915303 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.915046    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw" event={"ID":"28f92588-8375-4ef4-ae91-3ff783e71def","Type":"ContainerDied","Data":"eab7676579c60a3b08154b8005b5e8d2dab80b78ea57e73357dc12e4d52d191b"}
Nov 03 23:06:08.915303 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.915063    5647 scope.go:122] "RemoveContainer" containerID="f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57"
Nov 03 23:06:08.916444 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.916424    5647 generic.go:356] "Generic (PLEG): container finished" podID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerID="cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad" exitCode=0
Nov 03 23:06:08.916549 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.916459    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"
Nov 03 23:06:08.916549 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.916464    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" event={"ID":"445d4bb2-6baf-44ea-887d-4d0443b9fe01","Type":"ContainerDied","Data":"cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad"}
Nov 03 23:06:08.916549 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.916512    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp" event={"ID":"445d4bb2-6baf-44ea-887d-4d0443b9fe01","Type":"ContainerDied","Data":"33f73c2e920d83bb7452047d54c7b58ba5149a405ad8a5a88cb48c1071204281"}
Nov 03 23:06:08.924806 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.924788    5647 scope.go:122] "RemoveContainer" containerID="f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57"
Nov 03 23:06:08.925044 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:08.925026    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57\": container with ID starting with f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57 not found: ID does not exist" containerID="f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57"
Nov 03 23:06:08.925106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.925050    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57"} err="failed to get container status \"f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57\": rpc error: code = NotFound desc = could not find container \"f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57\": container with ID starting with f54701667edc185bbc5c08fb484c3ad973757e058b26f9ab1df933258c6ccd57 not found: ID does not exist"
Nov 03 23:06:08.925106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.925066    5647 scope.go:122] "RemoveContainer" containerID="cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad"
Nov 03 23:06:08.938250 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.938227    5647 scope.go:122] "RemoveContainer" containerID="cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad"
Nov 03 23:06:08.938583 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:08.938560    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad\": container with ID starting with cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad not found: ID does not exist" containerID="cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad"
Nov 03 23:06:08.938744 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:08.938593    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad"} err="failed to get container status \"cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad\": rpc error: code = NotFound desc = could not find container \"cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad\": container with ID starting with cb8090fec132a682ef412e6d8d7a9f0594f7ead09349e4cdd86f4f9c543241ad not found: ID does not exist"
Nov 03 23:06:09.078846 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:09.078809    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"]
Nov 03 23:06:09.082065 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:09.082034    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/simulator-w6-7540da-kserve-5567d5c5d8-8ncnw"]
Nov 03 23:06:09.189866 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:09.189780    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"]
Nov 03 23:06:09.193268 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:09.193243    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-7540da/premium-w6-7540da-kserve-6bb5c64df7-7fxvp"]
Nov 03 23:06:10.695995 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:10.695959    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" path="/var/lib/kubelet/pods/28f92588-8375-4ef4-ae91-3ff783e71def/volumes"
Nov 03 23:06:10.696396 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:10.696383    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" path="/var/lib/kubelet/pods/445d4bb2-6baf-44ea-887d-4d0443b9fe01/volumes"
Nov 03 23:06:16.043099 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:16.043060    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:06:16.563095 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:16.563058    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:06:35.248447 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:35.248397    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"]
Nov 03 23:06:35.249037 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:35.248607    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy" containerID="cri-o://9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0" gracePeriod=30
Nov 03 23:06:35.506231 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:35.506150    5647 patch_prober.go:29] interesting pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:06:35.506231 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:35.506212    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:06:40.518988 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.518930    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:06:40.596970 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.596940    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-envoy\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.597118 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597003    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-data\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.597118 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597030    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d206c63-b944-4b5b-8952-102b185b6925-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d206c63-b944-4b5b-8952-102b185b6925-istio-podinfo\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.597118 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597096    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-certs\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.597289 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597268    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-data" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:40.597342 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597274    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-credential-socket\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.597376 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597361    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d206c63-b944-4b5b-8952-102b185b6925-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d206c63-b944-4b5b-8952-102b185b6925-istiod-ca-cert\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.597454 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597435    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-certs" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:40.597549 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597530    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-socket\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.597609 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597547    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-credential-socket" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:40.597799 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597773    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d206c63-b944-4b5b-8952-102b185b6925-istiod-ca-cert" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:06:40.597799 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597615    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-istio-token\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-istio-token\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.597799 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597683    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-socket" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:40.598027 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.597871    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-kube-api-access-vkkq6\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-kube-api-access-vkkq6\") pod \"4d206c63-b944-4b5b-8952-102b185b6925\" (UID: \"4d206c63-b944-4b5b-8952-102b185b6925\") "
Nov 03 23:06:40.598426 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.598406    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:40.598468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.598434    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:40.598468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.598450    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:40.598468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.598465    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d206c63-b944-4b5b-8952-102b185b6925-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:40.598632 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.598478    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:40.599464 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.599444    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d206c63-b944-4b5b-8952-102b185b6925-istio-podinfo" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:06:40.599464 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.599444    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-envoy" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:40.599995 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.599974    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-istio-token" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:40.600304 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.600286    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-kube-api-access-vkkq6" pod "4d206c63-b944-4b5b-8952-102b185b6925" (UID: "4d206c63-b944-4b5b-8952-102b185b6925"). InnerVolumeSpecName "kube-api-access-vkkq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:40.699872 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.699838    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:40.699872 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.699874    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkkq6\" (UniqueName: \"kubernetes.io/projected/4d206c63-b944-4b5b-8952-102b185b6925-kube-api-access-vkkq6\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:40.700091 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.699890    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d206c63-b944-4b5b-8952-102b185b6925-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:40.700091 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:40.699903    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d206c63-b944-4b5b-8952-102b185b6925-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:41.157363 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.157328    5647 generic.go:356] "Generic (PLEG): container finished" podID="4d206c63-b944-4b5b-8952-102b185b6925" containerID="9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0" exitCode=0
Nov 03 23:06:41.157564 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.157386    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"
Nov 03 23:06:41.157564 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.157405    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" event={"ID":"4d206c63-b944-4b5b-8952-102b185b6925","Type":"ContainerDied","Data":"9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0"}
Nov 03 23:06:41.157564 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.157463    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq" event={"ID":"4d206c63-b944-4b5b-8952-102b185b6925","Type":"ContainerDied","Data":"278542c1353af30a377d625020337417be411a00c0f3466d2513ace5fee6015f"}
Nov 03 23:06:41.157564 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.157484    5647 scope.go:122] "RemoveContainer" containerID="9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0"
Nov 03 23:06:41.167512 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.167494    5647 scope.go:122] "RemoveContainer" containerID="8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7"
Nov 03 23:06:41.176539 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.176519    5647 scope.go:122] "RemoveContainer" containerID="9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0"
Nov 03 23:06:41.176813 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:41.176795    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0\": container with ID starting with 9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0 not found: ID does not exist" containerID="9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0"
Nov 03 23:06:41.176885 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.176820    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0"} err="failed to get container status \"9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0\": rpc error: code = NotFound desc = could not find container \"9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0\": container with ID starting with 9fcecf7e4683444b8e911cec2dce4d77abeb4a1f72269558b0763b130733fad0 not found: ID does not exist"
Nov 03 23:06:41.176885 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.176837    5647 scope.go:122] "RemoveContainer" containerID="8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7"
Nov 03 23:06:41.177090 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:41.177076    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7\": container with ID starting with 8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7 not found: ID does not exist" containerID="8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7"
Nov 03 23:06:41.177138 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.177094    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7"} err="failed to get container status \"8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7\": rpc error: code = NotFound desc = could not find container \"8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7\": container with ID starting with 8e3d3ae674619b834ddd1dfabf89326cf1750778d2248c367c9078e4166a09a7 not found: ID does not exist"
Nov 03 23:06:41.321311 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.321280    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"]
Nov 03 23:06:41.370707 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:41.370653    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq"]
Nov 03 23:06:42.697645 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:42.697610    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d206c63-b944-4b5b-8952-102b185b6925" path="/var/lib/kubelet/pods/4d206c63-b944-4b5b-8952-102b185b6925/volumes"
Nov 03 23:06:43.972978 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:43.972946    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:06:46.042223 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:46.042182    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:06:49.117996 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.117961    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"]
Nov 03 23:06:49.118802 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.118768    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr" podUID="2b03046d-8c44-4dce-9288-f252151ec2e1" containerName="maas-api" containerID="cri-o://d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da" gracePeriod=30
Nov 03 23:06:49.450278 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.450253    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:06:49.570712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.568126    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b03046d-8c44-4dce-9288-f252151ec2e1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b03046d-8c44-4dce-9288-f252151ec2e1-openshift-service-ca\") pod \"2b03046d-8c44-4dce-9288-f252151ec2e1\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") "
Nov 03 23:06:49.570712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.568251    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b03046d-8c44-4dce-9288-f252151ec2e1-kube-api-access-2zqst\" (UniqueName: \"kubernetes.io/projected/2b03046d-8c44-4dce-9288-f252151ec2e1-kube-api-access-2zqst\") pod \"2b03046d-8c44-4dce-9288-f252151ec2e1\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") "
Nov 03 23:06:49.570712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.568295    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls\") pod \"2b03046d-8c44-4dce-9288-f252151ec2e1\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") "
Nov 03 23:06:49.570712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.568353    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-metrics-certs\") pod \"2b03046d-8c44-4dce-9288-f252151ec2e1\" (UID: \"2b03046d-8c44-4dce-9288-f252151ec2e1\") "
Nov 03 23:06:49.576243 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.576209    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b03046d-8c44-4dce-9288-f252151ec2e1-openshift-service-ca" pod "2b03046d-8c44-4dce-9288-f252151ec2e1" (UID: "2b03046d-8c44-4dce-9288-f252151ec2e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:06:49.583832 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.583799    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls" pod "2b03046d-8c44-4dce-9288-f252151ec2e1" (UID: "2b03046d-8c44-4dce-9288-f252151ec2e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:06:49.598244 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.593565    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b03046d-8c44-4dce-9288-f252151ec2e1-kube-api-access-2zqst" pod "2b03046d-8c44-4dce-9288-f252151ec2e1" (UID: "2b03046d-8c44-4dce-9288-f252151ec2e1"). InnerVolumeSpecName "kube-api-access-2zqst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:49.598244 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.598128    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-metrics-certs" pod "2b03046d-8c44-4dce-9288-f252151ec2e1" (UID: "2b03046d-8c44-4dce-9288-f252151ec2e1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:06:49.670310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.670281    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zqst\" (UniqueName: \"kubernetes.io/projected/2b03046d-8c44-4dce-9288-f252151ec2e1-kube-api-access-2zqst\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:49.670310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.670308    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:49.670310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.670319    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b03046d-8c44-4dce-9288-f252151ec2e1-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:49.670538 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:49.670328    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b03046d-8c44-4dce-9288-f252151ec2e1-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:50.249991 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.249955    5647 generic.go:356] "Generic (PLEG): container finished" podID="2b03046d-8c44-4dce-9288-f252151ec2e1" containerID="d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da" exitCode=0
Nov 03 23:06:50.250473 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.250022    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr" event={"ID":"2b03046d-8c44-4dce-9288-f252151ec2e1","Type":"ContainerDied","Data":"d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da"}
Nov 03 23:06:50.250473 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.250074    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"
Nov 03 23:06:50.250473 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.250091    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr" event={"ID":"2b03046d-8c44-4dce-9288-f252151ec2e1","Type":"ContainerDied","Data":"bf4f788a896a12fc56a2059675ec5a1923764fe472cc52084a742f5ca17c342a"}
Nov 03 23:06:50.250473 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.250125    5647 scope.go:122] "RemoveContainer" containerID="d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da"
Nov 03 23:06:50.261235 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.261218    5647 scope.go:122] "RemoveContainer" containerID="d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da"
Nov 03 23:06:50.261521 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:50.261502    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da\": container with ID starting with d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da not found: ID does not exist" containerID="d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da"
Nov 03 23:06:50.261573 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.261529    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da"} err="failed to get container status \"d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da\": rpc error: code = NotFound desc = could not find container \"d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da\": container with ID starting with d282980464d35e868dfec07b31b838fd9e043b4463c27b95f44a2e1023e695da not found: ID does not exist"
Nov 03 23:06:50.413042 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.413010    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"]
Nov 03 23:06:50.462297 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.462265    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1048fc-7d769985dc-dqvlr"]
Nov 03 23:06:50.696426 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:50.696340    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b03046d-8c44-4dce-9288-f252151ec2e1" path="/var/lib/kubelet/pods/2b03046d-8c44-4dce-9288-f252151ec2e1/volumes"
Nov 03 23:06:51.936831 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:51.936791    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"]
Nov 03 23:06:51.937331 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:51.937052    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy" containerID="cri-o://0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7" gracePeriod=30
Nov 03 23:06:55.519076 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:55.519043    5647 patch_prober.go:29] interesting pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:06:55.519458 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:55.519098    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:06:57.212066 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.212040    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 23:06:57.294096 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.294065    5647 generic.go:356] "Generic (PLEG): container finished" podID="30be5938-7f33-495a-8537-a012b282e83a" containerID="0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7" exitCode=0
Nov 03 23:06:57.294285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.294134    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"
Nov 03 23:06:57.294285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.294165    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" event={"ID":"30be5938-7f33-495a-8537-a012b282e83a","Type":"ContainerDied","Data":"0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7"}
Nov 03 23:06:57.294285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.294229    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb" event={"ID":"30be5938-7f33-495a-8537-a012b282e83a","Type":"ContainerDied","Data":"e02ba448dbe67bb4e6cc38e89737507c615fcecd53cc626568aca793c4e9084e"}
Nov 03 23:06:57.294285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.294260    5647 scope.go:122] "RemoveContainer" containerID="0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7"
Nov 03 23:06:57.306977 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.306944    5647 scope.go:122] "RemoveContainer" containerID="353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9"
Nov 03 23:06:57.316225 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.316207    5647 scope.go:122] "RemoveContainer" containerID="0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7"
Nov 03 23:06:57.316532 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:57.316511    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7\": container with ID starting with 0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7 not found: ID does not exist" containerID="0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7"
Nov 03 23:06:57.316593 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.316542    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7"} err="failed to get container status \"0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7\": rpc error: code = NotFound desc = could not find container \"0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7\": container with ID starting with 0176893dac45a30660006987b6c27086191276305476662c1f9624a94efbf8c7 not found: ID does not exist"
Nov 03 23:06:57.316593 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.316563    5647 scope.go:122] "RemoveContainer" containerID="353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9"
Nov 03 23:06:57.316849 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:06:57.316829    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9\": container with ID starting with 353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9 not found: ID does not exist" containerID="353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9"
Nov 03 23:06:57.316907 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.316860    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9"} err="failed to get container status \"353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9\": rpc error: code = NotFound desc = could not find container \"353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9\": container with ID starting with 353f4e6cb9fff1c940ea1d7bb06197ce67ea36c8afe23d9e6e8393299083f8c9 not found: ID does not exist"
Nov 03 23:06:57.363523 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363489    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-certs\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.363523 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363525    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-envoy\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.363818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363581    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-kube-api-access-kk55j\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-kube-api-access-kk55j\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.363818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363605    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-data\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.363818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363706    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-istio-token\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-istio-token\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.363818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363789    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-certs" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:57.363818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363815    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-credential-socket\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.364094 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363863    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/30be5938-7f33-495a-8537-a012b282e83a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30be5938-7f33-495a-8537-a012b282e83a-istio-podinfo\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.364094 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363907    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-socket\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.364094 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363940    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30be5938-7f33-495a-8537-a012b282e83a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30be5938-7f33-495a-8537-a012b282e83a-istiod-ca-cert\") pod \"30be5938-7f33-495a-8537-a012b282e83a\" (UID: \"30be5938-7f33-495a-8537-a012b282e83a\") "
Nov 03 23:06:57.364094 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.363949    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-data" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:57.364094 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.364056    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-credential-socket" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:57.364524 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.364118    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-socket" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:57.364524 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.364360    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30be5938-7f33-495a-8537-a012b282e83a-istiod-ca-cert" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:06:57.364524 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.364440    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.364722 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.364523    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.364722 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.364543    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.364722 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.364560    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.365978 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.365957    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-istio-token" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:57.366081 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.366031    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-envoy" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:06:57.366563 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.366531    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-kube-api-access-kk55j" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "kube-api-access-kk55j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:06:57.366655 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.366555    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30be5938-7f33-495a-8537-a012b282e83a-istio-podinfo" pod "30be5938-7f33-495a-8537-a012b282e83a" (UID: "30be5938-7f33-495a-8537-a012b282e83a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:06:57.465638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.465567    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk55j\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-kube-api-access-kk55j\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.465638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.465591    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30be5938-7f33-495a-8537-a012b282e83a-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.465638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.465600    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30be5938-7f33-495a-8537-a012b282e83a-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.465638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.465608    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30be5938-7f33-495a-8537-a012b282e83a-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.465638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.465616    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30be5938-7f33-495a-8537-a012b282e83a-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:06:57.714298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.714259    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"]
Nov 03 23:06:57.724020 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:57.723938    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb"]
Nov 03 23:06:58.695938 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:06:58.695904    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30be5938-7f33-495a-8537-a012b282e83a" path="/var/lib/kubelet/pods/30be5938-7f33-495a-8537-a012b282e83a/volumes"
Nov 03 23:07:03.208369 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:03.208327    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"]
Nov 03 23:07:03.315951 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:03.315919    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"]
Nov 03 23:07:08.219716 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.213483    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn" podUID="61084784-f3d5-4c15-83ad-447cb2965488" containerName="payload-processing" containerID="cri-o://0b34d730505bbdd8120330c809021daae0ec11d257047690631c87ac72f2a858" gracePeriod=25
Nov 03 23:07:08.317134 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.317072    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm" podUID="fe35d9d5-26cd-4f57-9965-1be3714dbdd4" containerName="payload-pre-processing" containerID="cri-o://58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a" gracePeriod=25
Nov 03 23:07:08.366353 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.366318    5647 generic.go:356] "Generic (PLEG): container finished" podID="61084784-f3d5-4c15-83ad-447cb2965488" containerID="0b34d730505bbdd8120330c809021daae0ec11d257047690631c87ac72f2a858" exitCode=0
Nov 03 23:07:08.366529 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.366398    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn" event={"ID":"61084784-f3d5-4c15-83ad-447cb2965488","Type":"ContainerDied","Data":"0b34d730505bbdd8120330c809021daae0ec11d257047690631c87ac72f2a858"}
Nov 03 23:07:08.487574 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.487507    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 23:07:08.601158 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.601131    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 23:07:08.608050 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.608023    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61084784-f3d5-4c15-83ad-447cb2965488-kube-api-access-szt4v\" (UniqueName: \"kubernetes.io/projected/61084784-f3d5-4c15-83ad-447cb2965488-kube-api-access-szt4v\") pod \"61084784-f3d5-4c15-83ad-447cb2965488\" (UID: \"61084784-f3d5-4c15-83ad-447cb2965488\") "
Nov 03 23:07:08.608202 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.608169    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61084784-f3d5-4c15-83ad-447cb2965488-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61084784-f3d5-4c15-83ad-447cb2965488-plugins-config-volume\") pod \"61084784-f3d5-4c15-83ad-447cb2965488\" (UID: \"61084784-f3d5-4c15-83ad-447cb2965488\") "
Nov 03 23:07:08.608569 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.608547    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61084784-f3d5-4c15-83ad-447cb2965488-plugins-config-volume" pod "61084784-f3d5-4c15-83ad-447cb2965488" (UID: "61084784-f3d5-4c15-83ad-447cb2965488"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:08.608709 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.608684    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61084784-f3d5-4c15-83ad-447cb2965488-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:08.610348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.610326    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61084784-f3d5-4c15-83ad-447cb2965488-kube-api-access-szt4v" pod "61084784-f3d5-4c15-83ad-447cb2965488" (UID: "61084784-f3d5-4c15-83ad-447cb2965488"). InnerVolumeSpecName "kube-api-access-szt4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:08.709397 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.709373    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-plugins-config-volume\") pod \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\" (UID: \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\") "
Nov 03 23:07:08.709563 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.709547    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-kube-api-access-t9mkg\" (UniqueName: \"kubernetes.io/projected/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-kube-api-access-t9mkg\") pod \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\" (UID: \"fe35d9d5-26cd-4f57-9965-1be3714dbdd4\") "
Nov 03 23:07:08.709776 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.709753    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-plugins-config-volume" pod "fe35d9d5-26cd-4f57-9965-1be3714dbdd4" (UID: "fe35d9d5-26cd-4f57-9965-1be3714dbdd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:08.709963 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.709946    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:08.710019 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.709969    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szt4v\" (UniqueName: \"kubernetes.io/projected/61084784-f3d5-4c15-83ad-447cb2965488-kube-api-access-szt4v\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:08.711626 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.711604    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-kube-api-access-t9mkg" pod "fe35d9d5-26cd-4f57-9965-1be3714dbdd4" (UID: "fe35d9d5-26cd-4f57-9965-1be3714dbdd4"). InnerVolumeSpecName "kube-api-access-t9mkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:08.811265 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:08.811181    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9mkg\" (UniqueName: \"kubernetes.io/projected/fe35d9d5-26cd-4f57-9965-1be3714dbdd4-kube-api-access-t9mkg\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:09.372916 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.372881    5647 generic.go:356] "Generic (PLEG): container finished" podID="fe35d9d5-26cd-4f57-9965-1be3714dbdd4" containerID="58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a" exitCode=0
Nov 03 23:07:09.373348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.372932    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"
Nov 03 23:07:09.373348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.372965    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm" event={"ID":"fe35d9d5-26cd-4f57-9965-1be3714dbdd4","Type":"ContainerDied","Data":"58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a"}
Nov 03 23:07:09.373348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.373017    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm" event={"ID":"fe35d9d5-26cd-4f57-9965-1be3714dbdd4","Type":"ContainerDied","Data":"5e2d79bcbc0723b7169389c829030e513f649f17c2ba1de448a7a3a4bfee94b2"}
Nov 03 23:07:09.373348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.373049    5647 scope.go:122] "RemoveContainer" containerID="58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a"
Nov 03 23:07:09.374603 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.374585    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"
Nov 03 23:07:09.374603 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.374593    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn" event={"ID":"61084784-f3d5-4c15-83ad-447cb2965488","Type":"ContainerDied","Data":"99726caf57086158ab372d35d6a23c58f7f8508f0eb14ddd281dacdc4350a655"}
Nov 03 23:07:09.387293 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.387271    5647 scope.go:122] "RemoveContainer" containerID="58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a"
Nov 03 23:07:09.387600 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:09.387581    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a\": container with ID starting with 58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a not found: ID does not exist" containerID="58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a"
Nov 03 23:07:09.387650 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.387607    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a"} err="failed to get container status \"58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a\": rpc error: code = NotFound desc = could not find container \"58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a\": container with ID starting with 58a9a091c30ccd932fdd16f3918bff2736da5dc6a92c2de78e696d87dbfb7d7a not found: ID does not exist"
Nov 03 23:07:09.387650 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.387623    5647 scope.go:122] "RemoveContainer" containerID="0b34d730505bbdd8120330c809021daae0ec11d257047690631c87ac72f2a858"
Nov 03 23:07:09.574723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.574623    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"]
Nov 03 23:07:09.624613 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.624581    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm"]
Nov 03 23:07:09.692352 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.692322    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"]
Nov 03 23:07:09.696998 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:09.696974    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn"]
Nov 03 23:07:10.610599 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:10.608616    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"]
Nov 03 23:07:10.696391 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:10.696356    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61084784-f3d5-4c15-83ad-447cb2965488" path="/var/lib/kubelet/pods/61084784-f3d5-4c15-83ad-447cb2965488/volumes"
Nov 03 23:07:10.696894 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:10.696877    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe35d9d5-26cd-4f57-9965-1be3714dbdd4" path="/var/lib/kubelet/pods/fe35d9d5-26cd-4f57-9965-1be3714dbdd4/volumes"
Nov 03 23:07:10.713462 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:10.713433    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"]
Nov 03 23:07:15.609736 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.609697    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd" podUID="a81b3584-43d1-4ace-91fd-b729b7a0112e" containerName="payload-processing" containerID="cri-o://0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8" gracePeriod=25
Nov 03 23:07:15.713905 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.713863    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk" podUID="cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7" containerName="payload-pre-processing" containerID="cri-o://95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670" gracePeriod=25
Nov 03 23:07:15.890402 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.890369    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:07:15.923409 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.923369    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a81b3584-43d1-4ace-91fd-b729b7a0112e-kube-api-access-2gvsx\" (UniqueName: \"kubernetes.io/projected/a81b3584-43d1-4ace-91fd-b729b7a0112e-kube-api-access-2gvsx\") pod \"a81b3584-43d1-4ace-91fd-b729b7a0112e\" (UID: \"a81b3584-43d1-4ace-91fd-b729b7a0112e\") "
Nov 03 23:07:15.923612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.923453    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a81b3584-43d1-4ace-91fd-b729b7a0112e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81b3584-43d1-4ace-91fd-b729b7a0112e-plugins-config-volume\") pod \"a81b3584-43d1-4ace-91fd-b729b7a0112e\" (UID: \"a81b3584-43d1-4ace-91fd-b729b7a0112e\") "
Nov 03 23:07:15.930698 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.924316    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81b3584-43d1-4ace-91fd-b729b7a0112e-plugins-config-volume" pod "a81b3584-43d1-4ace-91fd-b729b7a0112e" (UID: "a81b3584-43d1-4ace-91fd-b729b7a0112e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:15.930698 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.924871    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81b3584-43d1-4ace-91fd-b729b7a0112e-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:15.938477 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.933311    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81b3584-43d1-4ace-91fd-b729b7a0112e-kube-api-access-2gvsx" pod "a81b3584-43d1-4ace-91fd-b729b7a0112e" (UID: "a81b3584-43d1-4ace-91fd-b729b7a0112e"). InnerVolumeSpecName "kube-api-access-2gvsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:15.983470 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:15.983444    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:07:16.026120 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.026089    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-plugins-config-volume\") pod \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\" (UID: \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\") "
Nov 03 23:07:16.026310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.026258    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-kube-api-access-njntg\" (UniqueName: \"kubernetes.io/projected/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-kube-api-access-njntg\") pod \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\" (UID: \"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7\") "
Nov 03 23:07:16.026563 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.026526    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-plugins-config-volume" pod "cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7" (UID: "cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:16.026779 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.026756    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:16.026872 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.026786    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gvsx\" (UniqueName: \"kubernetes.io/projected/a81b3584-43d1-4ace-91fd-b729b7a0112e-kube-api-access-2gvsx\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:16.028324 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.028304    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-kube-api-access-njntg" pod "cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7" (UID: "cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7"). InnerVolumeSpecName "kube-api-access-njntg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:16.048785 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:16.048744    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:07:16.128520 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.128447    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njntg\" (UniqueName: \"kubernetes.io/projected/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7-kube-api-access-njntg\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:16.429923 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.429802    5647 generic.go:356] "Generic (PLEG): container finished" podID="cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7" containerID="95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670" exitCode=0
Nov 03 23:07:16.429923 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.429861    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"
Nov 03 23:07:16.429923 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.429896    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk" event={"ID":"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7","Type":"ContainerDied","Data":"95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670"}
Nov 03 23:07:16.430198 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.429977    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk" event={"ID":"cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7","Type":"ContainerDied","Data":"63f608a5486f32a7fed1ccadc204d30ca91d88739879f1a0ad4f279e02c29d50"}
Nov 03 23:07:16.430198 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.430002    5647 scope.go:122] "RemoveContainer" containerID="95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670"
Nov 03 23:07:16.431415 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.431384    5647 generic.go:356] "Generic (PLEG): container finished" podID="a81b3584-43d1-4ace-91fd-b729b7a0112e" containerID="0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8" exitCode=0
Nov 03 23:07:16.431519 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.431444    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"
Nov 03 23:07:16.431519 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.431450    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd" event={"ID":"a81b3584-43d1-4ace-91fd-b729b7a0112e","Type":"ContainerDied","Data":"0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8"}
Nov 03 23:07:16.431519 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.431487    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd" event={"ID":"a81b3584-43d1-4ace-91fd-b729b7a0112e","Type":"ContainerDied","Data":"bb5362feca703942f964d494c7fdcb1a68b6b5f1f8e148de24a39350afcf2185"}
Nov 03 23:07:16.441945 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.441920    5647 scope.go:122] "RemoveContainer" containerID="95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670"
Nov 03 23:07:16.442351 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:16.442327    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670\": container with ID starting with 95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670 not found: ID does not exist" containerID="95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670"
Nov 03 23:07:16.442436 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.442362    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670"} err="failed to get container status \"95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670\": rpc error: code = NotFound desc = could not find container \"95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670\": container with ID starting with 95bfc9f5dbb4db09f4838749f337de6ec3874f4adbde13b69dc3d40c9fb6d670 not found: ID does not exist"
Nov 03 23:07:16.442436 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.442385    5647 scope.go:122] "RemoveContainer" containerID="0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8"
Nov 03 23:07:16.451616 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.451597    5647 scope.go:122] "RemoveContainer" containerID="0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8"
Nov 03 23:07:16.451917 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:16.451900    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8\": container with ID starting with 0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8 not found: ID does not exist" containerID="0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8"
Nov 03 23:07:16.451983 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.451927    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8"} err="failed to get container status \"0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8\": rpc error: code = NotFound desc = could not find container \"0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8\": container with ID starting with 0bd44e22b6787723869ae4c21c9b83c54956e68d309bc9139991f8f469265fa8 not found: ID does not exist"
Nov 03 23:07:16.614815 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.614781    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"]
Nov 03 23:07:16.623885 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.623856    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd"]
Nov 03 23:07:16.695943 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.695849    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a81b3584-43d1-4ace-91fd-b729b7a0112e" path="/var/lib/kubelet/pods/a81b3584-43d1-4ace-91fd-b729b7a0112e/volumes"
Nov 03 23:07:16.733298 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.733262    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"]
Nov 03 23:07:16.738272 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:16.738241    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk"]
Nov 03 23:07:18.695815 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:18.695783    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7" path="/var/lib/kubelet/pods/cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7/volumes"
Nov 03 23:07:22.696391 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:22.696359    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"]
Nov 03 23:07:22.696857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:22.696563    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f" podUID="09660488-c079-4e23-92d9-59c970651afe" containerName="maas-api" containerID="cri-o://596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6" gracePeriod=30
Nov 03 23:07:22.996254 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:22.996227    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 23:07:23.108029 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.107997    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/09660488-c079-4e23-92d9-59c970651afe-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09660488-c079-4e23-92d9-59c970651afe-openshift-service-ca\") pod \"09660488-c079-4e23-92d9-59c970651afe\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") "
Nov 03 23:07:23.108228 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.108098    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls\") pod \"09660488-c079-4e23-92d9-59c970651afe\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") "
Nov 03 23:07:23.108228 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.108123    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-metrics-certs\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-metrics-certs\") pod \"09660488-c079-4e23-92d9-59c970651afe\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") "
Nov 03 23:07:23.108228 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.108183    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09660488-c079-4e23-92d9-59c970651afe-kube-api-access-9qhwz\" (UniqueName: \"kubernetes.io/projected/09660488-c079-4e23-92d9-59c970651afe-kube-api-access-9qhwz\") pod \"09660488-c079-4e23-92d9-59c970651afe\" (UID: \"09660488-c079-4e23-92d9-59c970651afe\") "
Nov 03 23:07:23.108455 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.108427    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09660488-c079-4e23-92d9-59c970651afe-openshift-service-ca" pod "09660488-c079-4e23-92d9-59c970651afe" (UID: "09660488-c079-4e23-92d9-59c970651afe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:23.108587 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.108571    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09660488-c079-4e23-92d9-59c970651afe-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:23.110653 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.110622    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09660488-c079-4e23-92d9-59c970651afe-kube-api-access-9qhwz" pod "09660488-c079-4e23-92d9-59c970651afe" (UID: "09660488-c079-4e23-92d9-59c970651afe"). InnerVolumeSpecName "kube-api-access-9qhwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:23.110653 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.110637    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls" pod "09660488-c079-4e23-92d9-59c970651afe" (UID: "09660488-c079-4e23-92d9-59c970651afe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:23.110824 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.110772    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-metrics-certs" pod "09660488-c079-4e23-92d9-59c970651afe" (UID: "09660488-c079-4e23-92d9-59c970651afe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:23.210068 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.210036    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:23.210068 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.210062    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09660488-c079-4e23-92d9-59c970651afe-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:23.210068 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.210073    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qhwz\" (UniqueName: \"kubernetes.io/projected/09660488-c079-4e23-92d9-59c970651afe-kube-api-access-9qhwz\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:23.484632 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.484598    5647 generic.go:356] "Generic (PLEG): container finished" podID="09660488-c079-4e23-92d9-59c970651afe" containerID="596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6" exitCode=0
Nov 03 23:07:23.484827 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.484644    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"
Nov 03 23:07:23.484827 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.484697    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f" event={"ID":"09660488-c079-4e23-92d9-59c970651afe","Type":"ContainerDied","Data":"596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6"}
Nov 03 23:07:23.484827 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.484765    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f" event={"ID":"09660488-c079-4e23-92d9-59c970651afe","Type":"ContainerDied","Data":"4040a871c68625ad299a667eab084292b2ed196ba59d82e2128d464eaae571dc"}
Nov 03 23:07:23.484827 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.484787    5647 scope.go:122] "RemoveContainer" containerID="596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6"
Nov 03 23:07:23.495283 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.495250    5647 scope.go:122] "RemoveContainer" containerID="596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6"
Nov 03 23:07:23.495546 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:23.495524    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6\": container with ID starting with 596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6 not found: ID does not exist" containerID="596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6"
Nov 03 23:07:23.495623 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.495554    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6"} err="failed to get container status \"596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6\": rpc error: code = NotFound desc = could not find container \"596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6\": container with ID starting with 596e3ef9917144d33dc47c6b2d849a69c4cd0faf82a45f6965d48b555ef591a6 not found: ID does not exist"
Nov 03 23:07:23.647627 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.647596    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"]
Nov 03 23:07:23.651686 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.651649    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-7540da-6bdb464996-5qp6f"]
Nov 03 23:07:23.740398 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:23.740308    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:07:24.695660 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:24.695618    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09660488-c079-4e23-92d9-59c970651afe" path="/var/lib/kubelet/pods/09660488-c079-4e23-92d9-59c970651afe/volumes"
Nov 03 23:07:27.674535 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:27.674496    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"]
Nov 03 23:07:28.999720 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:28.999687    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"]
Nov 03 23:07:29.000140 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:28.999885    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb" podUID="36710485-2249-43c5-aa16-51f95289922d" containerName="maas-api" containerID="cri-o://2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa" gracePeriod=30
Nov 03 23:07:29.317118 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.317087    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 23:07:29.392266 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.392231    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36710485-2249-43c5-aa16-51f95289922d-kube-api-access-szqrn\" (UniqueName: \"kubernetes.io/projected/36710485-2249-43c5-aa16-51f95289922d-kube-api-access-szqrn\") pod \"36710485-2249-43c5-aa16-51f95289922d\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") "
Nov 03 23:07:29.393077 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.393042    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36710485-2249-43c5-aa16-51f95289922d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36710485-2249-43c5-aa16-51f95289922d-openshift-service-ca\") pod \"36710485-2249-43c5-aa16-51f95289922d\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") "
Nov 03 23:07:29.393256 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.393239    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-metrics-certs\") pod \"36710485-2249-43c5-aa16-51f95289922d\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") "
Nov 03 23:07:29.393451 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.393439    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls\") pod \"36710485-2249-43c5-aa16-51f95289922d\" (UID: \"36710485-2249-43c5-aa16-51f95289922d\") "
Nov 03 23:07:29.397518 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.395451    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36710485-2249-43c5-aa16-51f95289922d-openshift-service-ca" pod "36710485-2249-43c5-aa16-51f95289922d" (UID: "36710485-2249-43c5-aa16-51f95289922d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:29.397726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.397643    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-metrics-certs" pod "36710485-2249-43c5-aa16-51f95289922d" (UID: "36710485-2249-43c5-aa16-51f95289922d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:29.399498 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.399456    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36710485-2249-43c5-aa16-51f95289922d-kube-api-access-szqrn" pod "36710485-2249-43c5-aa16-51f95289922d" (UID: "36710485-2249-43c5-aa16-51f95289922d"). InnerVolumeSpecName "kube-api-access-szqrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:29.405470 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.405434    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls" pod "36710485-2249-43c5-aa16-51f95289922d" (UID: "36710485-2249-43c5-aa16-51f95289922d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:29.495531 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.495494    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:29.495531 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.495527    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szqrn\" (UniqueName: \"kubernetes.io/projected/36710485-2249-43c5-aa16-51f95289922d-kube-api-access-szqrn\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:29.495531 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.495537    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36710485-2249-43c5-aa16-51f95289922d-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:29.495780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.495546    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36710485-2249-43c5-aa16-51f95289922d-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:29.521573 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.521534    5647 generic.go:356] "Generic (PLEG): container finished" podID="36710485-2249-43c5-aa16-51f95289922d" containerID="2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa" exitCode=0
Nov 03 23:07:29.521766 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.521590    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"
Nov 03 23:07:29.521766 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.521615    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb" event={"ID":"36710485-2249-43c5-aa16-51f95289922d","Type":"ContainerDied","Data":"2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa"}
Nov 03 23:07:29.521766 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.521699    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb" event={"ID":"36710485-2249-43c5-aa16-51f95289922d","Type":"ContainerDied","Data":"dd8993029f8a494a247bf0a3f1552e834a27965e6aa90457fcd82d5e11dad1a6"}
Nov 03 23:07:29.521766 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.521726    5647 scope.go:122] "RemoveContainer" containerID="2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa"
Nov 03 23:07:29.532528 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.532441    5647 scope.go:122] "RemoveContainer" containerID="2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa"
Nov 03 23:07:29.533244 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:29.533215    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa\": container with ID starting with 2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa not found: ID does not exist" containerID="2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa"
Nov 03 23:07:29.533380 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.533254    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa"} err="failed to get container status \"2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa\": rpc error: code = NotFound desc = could not find container \"2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa\": container with ID starting with 2a5a815facc457c434d0372340d653471bf7b3c3ae13f14b0e1f1f3f216103fa not found: ID does not exist"
Nov 03 23:07:29.690942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.690860    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"]
Nov 03 23:07:29.695354 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:29.695328    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4a1480-bc7d5944f-qz8qb"]
Nov 03 23:07:30.695797 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:30.695764    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36710485-2249-43c5-aa16-51f95289922d" path="/var/lib/kubelet/pods/36710485-2249-43c5-aa16-51f95289922d/volumes"
Nov 03 23:07:32.675644 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:32.675583    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk" podUID="89fbd4d3-1c94-49e4-b81c-2f0fb8da3705" containerName="payload-pre-processing" containerID="cri-o://4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd" gracePeriod=25
Nov 03 23:07:32.935030 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:32.934960    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 23:07:33.039500 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.039465    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-kube-api-access-9gnd7\" (UniqueName: \"kubernetes.io/projected/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-kube-api-access-9gnd7\") pod \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\" (UID: \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\") "
Nov 03 23:07:33.039748 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.039605    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-plugins-config-volume\") pod \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\" (UID: \"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705\") "
Nov 03 23:07:33.040005 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.039980    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-plugins-config-volume" pod "89fbd4d3-1c94-49e4-b81c-2f0fb8da3705" (UID: "89fbd4d3-1c94-49e4-b81c-2f0fb8da3705"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:33.041884 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.041860    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-kube-api-access-9gnd7" pod "89fbd4d3-1c94-49e4-b81c-2f0fb8da3705" (UID: "89fbd4d3-1c94-49e4-b81c-2f0fb8da3705"). InnerVolumeSpecName "kube-api-access-9gnd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:33.140752 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.140715    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gnd7\" (UniqueName: \"kubernetes.io/projected/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-kube-api-access-9gnd7\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:33.140752 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.140751    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:33.549484 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.549453    5647 generic.go:356] "Generic (PLEG): container finished" podID="89fbd4d3-1c94-49e4-b81c-2f0fb8da3705" containerID="4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd" exitCode=0
Nov 03 23:07:33.549638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.549495    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"
Nov 03 23:07:33.549638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.549538    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk" event={"ID":"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705","Type":"ContainerDied","Data":"4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd"}
Nov 03 23:07:33.549638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.549597    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk" event={"ID":"89fbd4d3-1c94-49e4-b81c-2f0fb8da3705","Type":"ContainerDied","Data":"58d15305b1a44a58101faf747ce3b4c5d673a9b37d43774834d94190557e5126"}
Nov 03 23:07:33.549638 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.549622    5647 scope.go:122] "RemoveContainer" containerID="4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd"
Nov 03 23:07:33.560044 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.560021    5647 scope.go:122] "RemoveContainer" containerID="4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd"
Nov 03 23:07:33.560298 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:33.560281    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd\": container with ID starting with 4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd not found: ID does not exist" containerID="4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd"
Nov 03 23:07:33.560356 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.560308    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd"} err="failed to get container status \"4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd\": rpc error: code = NotFound desc = could not find container \"4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd\": container with ID starting with 4869d1f421d5dc6069d25477a266fe006a4c5d16b502f5a12a3b8134a271bbcd not found: ID does not exist"
Nov 03 23:07:33.712125 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.712086    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"]
Nov 03 23:07:33.761529 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:33.761496    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk"]
Nov 03 23:07:34.696468 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:34.696425    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89fbd4d3-1c94-49e4-b81c-2f0fb8da3705" path="/var/lib/kubelet/pods/89fbd4d3-1c94-49e4-b81c-2f0fb8da3705/volumes"
Nov 03 23:07:43.573754 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:43.573716    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"]
Nov 03 23:07:43.623169 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:43.623140    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"]
Nov 03 23:07:46.043320 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:46.043286    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:07:50.467231 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:50.467191    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"]
Nov 03 23:07:50.467631 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:50.467425    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" containerID="cri-o://ffb16ba850205ea68fb7cf264d7082b0be13104d55ebbac32685cce5e03c7ea6" gracePeriod=30
Nov 03 23:07:51.446166 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:51.446129    5647 patch_prober.go:29] interesting pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:07:51.446350 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:51.446184    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:07:53.809371 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:53.809334    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:07:55.679342 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.679308    5647 generic.go:356] "Generic (PLEG): container finished" podID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerID="ffb16ba850205ea68fb7cf264d7082b0be13104d55ebbac32685cce5e03c7ea6" exitCode=0
Nov 03 23:07:55.679772 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.679394    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" event={"ID":"ea7424a3-4b15-44af-beae-c41b3eb13801","Type":"ContainerDied","Data":"ffb16ba850205ea68fb7cf264d7082b0be13104d55ebbac32685cce5e03c7ea6"}
Nov 03 23:07:55.679772 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.679479    5647 scope.go:122] "RemoveContainer" containerID="ed3507663c10e41044028ea6347525817e38e35ae49788fbca0fe06741a978b8"
Nov 03 23:07:55.720983 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.720956    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 23:07:55.741755 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.741702    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-envoy\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.741755 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.741752    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea7424a3-4b15-44af-beae-c41b3eb13801-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7424a3-4b15-44af-beae-c41b3eb13801-istiod-ca-cert\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.741887 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.741792    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-socket\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.741887 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.741811    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-certs\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.741887 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.741840    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-token\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-token\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.742031 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.741895    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-kube-api-access-bgmbk\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-kube-api-access-bgmbk\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.742031 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.741931    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-data\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.742171 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.742095    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-certs" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:55.742171 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.742138    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-socket" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:55.742305 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.742186    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-data" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:55.742305 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.742186    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea7424a3-4b15-44af-beae-c41b3eb13801-istiod-ca-cert" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:07:55.742305 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.742198    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-credential-socket\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.742305 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.742241    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-podinfo\") pod \"ea7424a3-4b15-44af-beae-c41b3eb13801\" (UID: \"ea7424a3-4b15-44af-beae-c41b3eb13801\") "
Nov 03 23:07:55.742899 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.742605    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-credential-socket" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:55.743164 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.742992    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.743164 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.743015    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea7424a3-4b15-44af-beae-c41b3eb13801-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.743164 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.743031    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.743164 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.743045    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.743164 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.743060    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.744139 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.744115    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-envoy" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:55.744508 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.744491    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-podinfo" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:07:55.744790 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.744767    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-token" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:55.744884 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.744798    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-kube-api-access-bgmbk" pod "ea7424a3-4b15-44af-beae-c41b3eb13801" (UID: "ea7424a3-4b15-44af-beae-c41b3eb13801"). InnerVolumeSpecName "kube-api-access-bgmbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:55.844633 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.844599    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.844633 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.844627    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgmbk\" (UniqueName: \"kubernetes.io/projected/ea7424a3-4b15-44af-beae-c41b3eb13801-kube-api-access-bgmbk\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.844633 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.844639    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.844933 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.844648    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea7424a3-4b15-44af-beae-c41b3eb13801-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:55.996842 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:55.996760    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"]
Nov 03 23:07:56.686368 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:56.686338    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"
Nov 03 23:07:56.686830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:56.686338    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs" event={"ID":"ea7424a3-4b15-44af-beae-c41b3eb13801","Type":"ContainerDied","Data":"567f83b6fe6c96bae0fa34f799a222af045646df8f54315127d0a541a22ac093"}
Nov 03 23:07:56.686830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:56.686501    5647 scope.go:122] "RemoveContainer" containerID="ffb16ba850205ea68fb7cf264d7082b0be13104d55ebbac32685cce5e03c7ea6"
Nov 03 23:07:56.852620 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:56.852590    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"]
Nov 03 23:07:56.902043 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:56.902015    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs"]
Nov 03 23:07:57.108744 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:57.108718    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:07:57.117314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:57.117294    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:07:58.595578 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.595539    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" containerID="cri-o://bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5" gracePeriod=45
Nov 03 23:07:58.643121 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.643084    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" containerID="cri-o://eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9" gracePeriod=45
Nov 03 23:07:58.697072 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.697043    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" path="/var/lib/kubelet/pods/ea7424a3-4b15-44af-beae-c41b3eb13801/volumes"
Nov 03 23:07:58.836880 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.836839    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": dial tcp 10.134.0.56:8000: connect: connection refused"
Nov 03 23:07:58.909763 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.909734    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 23:07:58.934121 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.934098    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 23:07:58.975991 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.975963    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86774a7f-0cd5-43d6-9162-c9bf260453ed-tls-certs\" (UniqueName: \"kubernetes.io/secret/86774a7f-0cd5-43d6-9162-c9bf260453ed-tls-certs\") pod \"86774a7f-0cd5-43d6-9162-c9bf260453ed\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") "
Nov 03 23:07:58.976167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976022    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-model-cache\") pod \"86774a7f-0cd5-43d6-9162-c9bf260453ed\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") "
Nov 03 23:07:58.976167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976088    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-dshm\") pod \"7892a50d-3c45-481c-8c05-2a70be4d71b5\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") "
Nov 03 23:07:58.976167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976121    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-dshm\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-dshm\") pod \"86774a7f-0cd5-43d6-9162-c9bf260453ed\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") "
Nov 03 23:07:58.976167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976156    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-home\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-home\") pod \"7892a50d-3c45-481c-8c05-2a70be4d71b5\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") "
Nov 03 23:07:58.976399 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976185    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-model-cache\") pod \"7892a50d-3c45-481c-8c05-2a70be4d71b5\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") "
Nov 03 23:07:58.976399 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976215    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-home\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-home\") pod \"86774a7f-0cd5-43d6-9162-c9bf260453ed\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") "
Nov 03 23:07:58.976399 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976241    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7892a50d-3c45-481c-8c05-2a70be4d71b5-kube-api-access-lc6zz\" (UniqueName: \"kubernetes.io/projected/7892a50d-3c45-481c-8c05-2a70be4d71b5-kube-api-access-lc6zz\") pod \"7892a50d-3c45-481c-8c05-2a70be4d71b5\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") "
Nov 03 23:07:58.976399 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976267    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-tmp-dir\") pod \"7892a50d-3c45-481c-8c05-2a70be4d71b5\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") "
Nov 03 23:07:58.976399 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976329    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-tmp-dir\") pod \"86774a7f-0cd5-43d6-9162-c9bf260453ed\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") "
Nov 03 23:07:58.976399 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976384    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7892a50d-3c45-481c-8c05-2a70be4d71b5-tls-certs\" (UniqueName: \"kubernetes.io/secret/7892a50d-3c45-481c-8c05-2a70be4d71b5-tls-certs\") pod \"7892a50d-3c45-481c-8c05-2a70be4d71b5\" (UID: \"7892a50d-3c45-481c-8c05-2a70be4d71b5\") "
Nov 03 23:07:58.976788 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976482    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-home" pod "86774a7f-0cd5-43d6-9162-c9bf260453ed" (UID: "86774a7f-0cd5-43d6-9162-c9bf260453ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:58.976788 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976489    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86774a7f-0cd5-43d6-9162-c9bf260453ed-kube-api-access-mdwct\" (UniqueName: \"kubernetes.io/projected/86774a7f-0cd5-43d6-9162-c9bf260453ed-kube-api-access-mdwct\") pod \"86774a7f-0cd5-43d6-9162-c9bf260453ed\" (UID: \"86774a7f-0cd5-43d6-9162-c9bf260453ed\") "
Nov 03 23:07:58.976788 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976743    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-tmp-dir" pod "86774a7f-0cd5-43d6-9162-c9bf260453ed" (UID: "86774a7f-0cd5-43d6-9162-c9bf260453ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:58.977439 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.976838    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-home" pod "7892a50d-3c45-481c-8c05-2a70be4d71b5" (UID: "7892a50d-3c45-481c-8c05-2a70be4d71b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:58.977439 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.977114    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-model-cache" pod "7892a50d-3c45-481c-8c05-2a70be4d71b5" (UID: "7892a50d-3c45-481c-8c05-2a70be4d71b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:58.977439 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.977219    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-tmp-dir" pod "7892a50d-3c45-481c-8c05-2a70be4d71b5" (UID: "7892a50d-3c45-481c-8c05-2a70be4d71b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:58.977703 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.977483    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:58.977703 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.977504    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:58.977703 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.977517    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:58.977703 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.977584    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-model-cache" pod "86774a7f-0cd5-43d6-9162-c9bf260453ed" (UID: "86774a7f-0cd5-43d6-9162-c9bf260453ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:58.981446 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.981391    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86774a7f-0cd5-43d6-9162-c9bf260453ed-kube-api-access-mdwct" pod "86774a7f-0cd5-43d6-9162-c9bf260453ed" (UID: "86774a7f-0cd5-43d6-9162-c9bf260453ed"). InnerVolumeSpecName "kube-api-access-mdwct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:58.981446 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.981370    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86774a7f-0cd5-43d6-9162-c9bf260453ed-tls-certs" pod "86774a7f-0cd5-43d6-9162-c9bf260453ed" (UID: "86774a7f-0cd5-43d6-9162-c9bf260453ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:58.981446 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.981435    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7892a50d-3c45-481c-8c05-2a70be4d71b5-kube-api-access-lc6zz" pod "7892a50d-3c45-481c-8c05-2a70be4d71b5" (UID: "7892a50d-3c45-481c-8c05-2a70be4d71b5"). InnerVolumeSpecName "kube-api-access-lc6zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:07:58.981798 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.981772    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-dshm" pod "7892a50d-3c45-481c-8c05-2a70be4d71b5" (UID: "7892a50d-3c45-481c-8c05-2a70be4d71b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:58.981910 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.981816    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-dshm" pod "86774a7f-0cd5-43d6-9162-c9bf260453ed" (UID: "86774a7f-0cd5-43d6-9162-c9bf260453ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:07:58.981961 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:58.981914    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7892a50d-3c45-481c-8c05-2a70be4d71b5-tls-certs" pod "7892a50d-3c45-481c-8c05-2a70be4d71b5" (UID: "7892a50d-3c45-481c-8c05-2a70be4d71b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:07:59.078569 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078540    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.078569 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078569    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc6zz\" (UniqueName: \"kubernetes.io/projected/7892a50d-3c45-481c-8c05-2a70be4d71b5-kube-api-access-lc6zz\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.078569 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078579    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.078845 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078588    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7892a50d-3c45-481c-8c05-2a70be4d71b5-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.078845 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078596    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdwct\" (UniqueName: \"kubernetes.io/projected/86774a7f-0cd5-43d6-9162-c9bf260453ed-kube-api-access-mdwct\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.078845 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078605    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86774a7f-0cd5-43d6-9162-c9bf260453ed-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.078845 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078613    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.078845 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078621    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7892a50d-3c45-481c-8c05-2a70be4d71b5-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.078845 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.078629    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86774a7f-0cd5-43d6-9162-c9bf260453ed-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:07:59.704074 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.704041    5647 generic.go:356] "Generic (PLEG): container finished" podID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerID="bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5" exitCode=0
Nov 03 23:07:59.704559 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.704097    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"
Nov 03 23:07:59.704559 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.704127    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" event={"ID":"7892a50d-3c45-481c-8c05-2a70be4d71b5","Type":"ContainerDied","Data":"bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5"}
Nov 03 23:07:59.704559 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.704163    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" event={"ID":"7892a50d-3c45-481c-8c05-2a70be4d71b5","Type":"ContainerDied","Data":"6da66ad19c1cc3c60ed2319e8d2ba7c3952026aeaf67310229b74207208e636d"}
Nov 03 23:07:59.704559 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.704174    5647 scope.go:122] "RemoveContainer" containerID="bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5"
Nov 03 23:07:59.705643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.705621    5647 generic.go:356] "Generic (PLEG): container finished" podID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerID="eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9" exitCode=0
Nov 03 23:07:59.705760 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.705683    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"
Nov 03 23:07:59.705760 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.705724    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" event={"ID":"86774a7f-0cd5-43d6-9162-c9bf260453ed","Type":"ContainerDied","Data":"eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9"}
Nov 03 23:07:59.705760 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.705754    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt" event={"ID":"86774a7f-0cd5-43d6-9162-c9bf260453ed","Type":"ContainerDied","Data":"6150cbd07268f9e34e9686448b5922074858bebb29d59c0730a453ab48915577"}
Nov 03 23:07:59.714252 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.714230    5647 scope.go:122] "RemoveContainer" containerID="bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5"
Nov 03 23:07:59.714636 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:59.714523    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5\": container with ID starting with bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5 not found: ID does not exist" containerID="bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5"
Nov 03 23:07:59.714636 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.714549    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5"} err="failed to get container status \"bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5\": rpc error: code = NotFound desc = could not find container \"bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5\": container with ID starting with bd8d80d0db861f66889a54f2b1bfc184933038d43c547b5c7e66adce0979b4c5 not found: ID does not exist"
Nov 03 23:07:59.714636 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.714565    5647 scope.go:122] "RemoveContainer" containerID="eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9"
Nov 03 23:07:59.722947 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.722929    5647 scope.go:122] "RemoveContainer" containerID="eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9"
Nov 03 23:07:59.723224 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:07:59.723204    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9\": container with ID starting with eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9 not found: ID does not exist" containerID="eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9"
Nov 03 23:07:59.723290 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.723228    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9"} err="failed to get container status \"eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9\": rpc error: code = NotFound desc = could not find container \"eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9\": container with ID starting with eef899e7b994585732533144147be6a2c4e041d6a4c9b0cab219aee5de3d4dc9 not found: ID does not exist"
Nov 03 23:07:59.866792 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.866767    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"]
Nov 03 23:07:59.890184 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.890154    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:07:59.915852 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:07:59.915807    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/distinct-2-w2-164c36-kserve-6fbd7c645f-jwf8c"]
Nov 03 23:08:00.023144 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:00.023111    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"]
Nov 03 23:08:00.027223 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:00.027196    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-164c36/simulator-w2-164c36-kserve-bc8cd59d8-4djvt"]
Nov 03 23:08:00.695430 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:00.695394    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" path="/var/lib/kubelet/pods/7892a50d-3c45-481c-8c05-2a70be4d71b5/volumes"
Nov 03 23:08:00.695865 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:00.695850    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" path="/var/lib/kubelet/pods/86774a7f-0cd5-43d6-9162-c9bf260453ed/volumes"
Nov 03 23:08:04.345088 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.345047    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"]
Nov 03 23:08:04.345550 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.345258    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" containerID="cri-o://832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4" gracePeriod=30
Nov 03 23:08:04.394985 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.394921    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"]
Nov 03 23:08:04.395166 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.395142    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" containerID="cri-o://f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8" gracePeriod=30
Nov 03 23:08:04.396541 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.396519    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"]
Nov 03 23:08:04.396764 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.396735    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" containerID="cri-o://92432074c9316d7ffd93756d0965f9e1bc050b43adb38dfb74aca7c8f75944db" gracePeriod=30
Nov 03 23:08:04.450420 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.444226    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"]
Nov 03 23:08:04.450420 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.444528    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" containerID="cri-o://e5bd53059e60c8c3bfac9a0a378e25f4aa9af00935d554ebfb1fe485645c464b" gracePeriod=30
Nov 03 23:08:04.562156 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.562123    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-np57c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:08:04.562319 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:04.562188    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:08:06.213183 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:06.213148    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"]
Nov 03 23:08:07.152729 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:07.152653    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:08:07.152879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:07.152750    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:08:07.403783 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:07.403698    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:08:07.404218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:07.403777    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:08:08.464545 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.464506    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"]
Nov 03 23:08:08.465085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465028    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy"
Nov 03 23:08:08.465085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465042    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy"
Nov 03 23:08:08.465085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465054    5647 cpu_manager.go:414] "containerMap: removing container" podUID="09660488-c079-4e23-92d9-59c970651afe" containerName="maas-api"
Nov 03 23:08:08.465085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465059    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09660488-c079-4e23-92d9-59c970651afe" containerName="maas-api"
Nov 03 23:08:08.465085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465067    5647 cpu_manager.go:414] "containerMap: removing container" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy"
Nov 03 23:08:08.465085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465074    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy"
Nov 03 23:08:08.465085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465088    5647 cpu_manager.go:414] "containerMap: removing container" podUID="4e4940fe-818b-4b7a-a64d-33f8a8c0e954" containerName="payload-processing"
Nov 03 23:08:08.465085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465093    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e4940fe-818b-4b7a-a64d-33f8a8c0e954" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465100    5647 cpu_manager.go:414] "containerMap: removing container" podUID="4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" containerName="maas-api"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465104    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" containerName="maas-api"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465111    5647 cpu_manager.go:414] "containerMap: removing container" podUID="0417d2a7-b9ce-484c-89af-c7a80319b260" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465115    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0417d2a7-b9ce-484c-89af-c7a80319b260" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465123    5647 cpu_manager.go:414] "containerMap: removing container" podUID="2b03046d-8c44-4dce-9288-f252151ec2e1" containerName="maas-api"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465128    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b03046d-8c44-4dce-9288-f252151ec2e1" containerName="maas-api"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465140    5647 cpu_manager.go:414] "containerMap: removing container" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465144    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465153    5647 cpu_manager.go:414] "containerMap: removing container" podUID="fe35d9d5-26cd-4f57-9965-1be3714dbdd4" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465161    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe35d9d5-26cd-4f57-9965-1be3714dbdd4" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465173    5647 cpu_manager.go:414] "containerMap: removing container" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465177    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465184    5647 cpu_manager.go:414] "containerMap: removing container" podUID="3ca465ff-12ce-41df-8642-9b0cd07dc68b" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465189    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ca465ff-12ce-41df-8642-9b0cd07dc68b" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465194    5647 cpu_manager.go:414] "containerMap: removing container" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465198    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465203    5647 cpu_manager.go:414] "containerMap: removing container" podUID="bc98e8b3-e780-4bc2-bc7f-1db2f505becc" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465207    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc98e8b3-e780-4bc2-bc7f-1db2f505becc" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465214    5647 cpu_manager.go:414] "containerMap: removing container" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465218    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465224    5647 cpu_manager.go:414] "containerMap: removing container" podUID="67837717-e0d8-445d-82ae-7392f7a104c3" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465229    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67837717-e0d8-445d-82ae-7392f7a104c3" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465241    5647 cpu_manager.go:414] "containerMap: removing container" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465246    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465251    5647 cpu_manager.go:414] "containerMap: removing container" podUID="61084784-f3d5-4c15-83ad-447cb2965488" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465255    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61084784-f3d5-4c15-83ad-447cb2965488" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465261    5647 cpu_manager.go:414] "containerMap: removing container" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465267    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465275    5647 cpu_manager.go:414] "containerMap: removing container" podUID="cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465279    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465286    5647 cpu_manager.go:414] "containerMap: removing container" podUID="12830a1b-7618-41cf-860d-932a2ddd38a1" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465290    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12830a1b-7618-41cf-860d-932a2ddd38a1" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465295    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a81b3584-43d1-4ace-91fd-b729b7a0112e" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465300    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a81b3584-43d1-4ace-91fd-b729b7a0112e" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465309    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8576b4d8-ff39-4559-aa81-8570063bfe4a" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465317    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8576b4d8-ff39-4559-aa81-8570063bfe4a" containerName="payload-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465328    5647 cpu_manager.go:414] "containerMap: removing container" podUID="89fbd4d3-1c94-49e4-b81c-2f0fb8da3705" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465332    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89fbd4d3-1c94-49e4-b81c-2f0fb8da3705" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465339    5647 cpu_manager.go:414] "containerMap: removing container" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465343    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465349    5647 cpu_manager.go:414] "containerMap: removing container" podUID="36710485-2249-43c5-aa16-51f95289922d" containerName="maas-api"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465354    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36710485-2249-43c5-aa16-51f95289922d" containerName="maas-api"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465362    5647 cpu_manager.go:414] "containerMap: removing container" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465366    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465374    5647 cpu_manager.go:414] "containerMap: removing container" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465379    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465393    5647 cpu_manager.go:414] "containerMap: removing container" podUID="6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465398    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465406    5647 cpu_manager.go:414] "containerMap: removing container" podUID="987eea5f-e434-4274-99fe-b0213675ef77" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465411    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="987eea5f-e434-4274-99fe-b0213675ef77" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465417    5647 cpu_manager.go:414] "containerMap: removing container" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465421    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465427    5647 cpu_manager.go:414] "containerMap: removing container" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465431    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465437    5647 cpu_manager.go:414] "containerMap: removing container" podUID="5da35a7f-116b-4518-b27b-4fc35a4c7f9e" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465441    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5da35a7f-116b-4518-b27b-4fc35a4c7f9e" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465451    5647 cpu_manager.go:414] "containerMap: removing container" podUID="f62143b4-dae9-42b0-a6a1-e042d8e71257" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465455    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f62143b4-dae9-42b0-a6a1-e042d8e71257" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465465    5647 cpu_manager.go:414] "containerMap: removing container" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465473    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465482    5647 cpu_manager.go:414] "containerMap: removing container" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465487    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465495    5647 cpu_manager.go:414] "containerMap: removing container" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465499    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465505    5647 cpu_manager.go:414] "containerMap: removing container" podUID="b834e991-6ce5-4d77-84ca-b8f3a2225a84" containerName="payload-pre-processing"
Nov 03 23:08:08.465497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465509    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b834e991-6ce5-4d77-84ca-b8f3a2225a84" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465574    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="8576b4d8-ff39-4559-aa81-8570063bfe4a" containerName="payload-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465584    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="89fbd4d3-1c94-49e4-b81c-2f0fb8da3705" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465591    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465596    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="36710485-2249-43c5-aa16-51f95289922d" containerName="maas-api"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465601    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="7892a50d-3c45-481c-8c05-2a70be4d71b5" containerName="main"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465607    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465612    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c9ed8a2-ef4f-41f1-a896-bf1cd7bf1198" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465621    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="987eea5f-e434-4274-99fe-b0213675ef77" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465629    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="86774a7f-0cd5-43d6-9162-c9bf260453ed" containerName="main"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465636    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="30be5938-7f33-495a-8537-a012b282e83a" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465641    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="5da35a7f-116b-4518-b27b-4fc35a4c7f9e" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465647    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="f62143b4-dae9-42b0-a6a1-e042d8e71257" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465652    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="83443efc-b020-45fb-a722-0e7eb4e5c719" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465656    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="44a7b927-3900-4a1f-9270-b72dd3239865" containerName="main"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465660    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="b834e991-6ce5-4d77-84ca-b8f3a2225a84" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465685    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="8dfa5234-0fb8-4b64-b263-313228dc237e" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465691    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="09660488-c079-4e23-92d9-59c970651afe" containerName="maas-api"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465697    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465702    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e4940fe-818b-4b7a-a64d-33f8a8c0e954" containerName="payload-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465707    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e81333e-ec9f-4f3a-8ff2-b8dc6d2bd863" containerName="maas-api"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465711    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="0417d2a7-b9ce-484c-89af-c7a80319b260" containerName="payload-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465716    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b03046d-8c44-4dce-9288-f252151ec2e1" containerName="maas-api"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465723    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465728    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="9406813d-528c-4d15-8276-7d13f292cb13" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465733    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe35d9d5-26cd-4f57-9965-1be3714dbdd4" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465739    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="445d4bb2-6baf-44ea-887d-4d0443b9fe01" containerName="main"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465744    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ca465ff-12ce-41df-8642-9b0cd07dc68b" containerName="payload-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465753    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="28f92588-8375-4ef4-ae91-3ff783e71def" containerName="main"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465762    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc98e8b3-e780-4bc2-bc7f-1db2f505becc" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465769    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ba930d7-6e52-45ac-a9c5-95e03546ee2d" containerName="main"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465774    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="67837717-e0d8-445d-82ae-7392f7a104c3" containerName="payload-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465780    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465785    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="61084784-f3d5-4c15-83ad-447cb2965488" containerName="payload-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465790    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdd92147-66a8-49de-9612-468a9c71cac1" containerName="istio-proxy"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465794    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf69f6de-21e1-40a8-a9cd-5e8a22e57ee7" containerName="payload-pre-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465800    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="12830a1b-7618-41cf-860d-932a2ddd38a1" containerName="payload-processing"
Nov 03 23:08:08.468879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.465805    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a81b3584-43d1-4ace-91fd-b729b7a0112e" containerName="payload-processing"
Nov 03 23:08:08.470917 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.470790    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.526006 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.525963    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"]
Nov 03 23:08:08.572233 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.572200    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 23:08:08.572396 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.572238    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p2s2g\""
Nov 03 23:08:08.616264 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.616232    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.616438 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.616358    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.616438 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.616413    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbq9r\" (UniqueName: \"kubernetes.io/projected/1538d524-6411-47f9-a8b6-fa97d897873a-kube-api-access-gbq9r\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.718205 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.718116    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.718383 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.718244    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.718383 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.718298    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbq9r\" (UniqueName: \"kubernetes.io/projected/1538d524-6411-47f9-a8b6-fa97d897873a-kube-api-access-gbq9r\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.718915 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.718890    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.720574 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.720549    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.779789 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.779747    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbq9r\" (UniqueName: \"kubernetes.io/projected/1538d524-6411-47f9-a8b6-fa97d897873a-kube-api-access-gbq9r\") pod \"maas-api-revoke-sub-f5b27b8f-c6xhj\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:08.784909 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:08.784888    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:09.161774 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.161707    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"]
Nov 03 23:08:09.164257 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:08:09.164232    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1538d524_6411_47f9_a8b6_fa97d897873a.slice/crio-f66d171ab826c78210b7d50e6371d4eb939ebdfc7e570f26ec34b084d3fadd1c WatchSource:0}: Error finding container f66d171ab826c78210b7d50e6371d4eb939ebdfc7e570f26ec34b084d3fadd1c: Status 404 returned error can't find the container with id f66d171ab826c78210b7d50e6371d4eb939ebdfc7e570f26ec34b084d3fadd1c
Nov 03 23:08:09.422365 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.422255    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:08:09.422365 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.422325    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:08:09.652070 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.651822    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 23:08:09.717387 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.717353    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 23:08:09.730933 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.730888    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-credential-socket\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731113 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.730973    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-certs\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731113 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731023    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-data\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731113 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731083    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-socket\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731296 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731138    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-kube-api-access-drb84\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-kube-api-access-drb84\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731296 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731209    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-credential-socket" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.731296 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731262    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-token\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-token\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731469 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731311    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-podinfo\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731469 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731361    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-envoy\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731469 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731398    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istiod-ca-cert\") pod \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\" (UID: \"12957ee7-e8d8-4e4d-8060-6cf8fc91c410\") "
Nov 03 23:08:09.731469 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731414    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-data" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.731778 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731655    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-certs" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.731851 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731818    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.731851 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731838    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.731929 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.731854    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.732177 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.732150    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-socket" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.732265 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.732243    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istiod-ca-cert" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:09.734825 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.734652    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-envoy" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.734825 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.734762    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-podinfo" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:09.738383 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.735801    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-kube-api-access-drb84" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "kube-api-access-drb84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:09.738383 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.735879    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-token" pod "12957ee7-e8d8-4e4d-8060-6cf8fc91c410" (UID: "12957ee7-e8d8-4e4d-8060-6cf8fc91c410"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:09.766587 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.766551    5647 generic.go:356] "Generic (PLEG): container finished" podID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerID="f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8" exitCode=0
Nov 03 23:08:09.766775 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.766597    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"
Nov 03 23:08:09.766775 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.766617    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" event={"ID":"81d99b28-c0aa-4312-9cd9-49fc570a7104","Type":"ContainerDied","Data":"f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8"}
Nov 03 23:08:09.766775 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.766709    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg" event={"ID":"81d99b28-c0aa-4312-9cd9-49fc570a7104","Type":"ContainerDied","Data":"dc96ff1667a9f6f5e92e0f297eb657da684437985280fb0c2b9fd4d4286940c0"}
Nov 03 23:08:09.766775 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.766741    5647 scope.go:122] "RemoveContainer" containerID="f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8"
Nov 03 23:08:09.768758 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.768734    5647 generic.go:356] "Generic (PLEG): container finished" podID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerID="92432074c9316d7ffd93756d0965f9e1bc050b43adb38dfb74aca7c8f75944db" exitCode=0
Nov 03 23:08:09.768913 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.768756    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" event={"ID":"66f55b8f-0627-455d-a540-3e02a6d25c01","Type":"ContainerDied","Data":"92432074c9316d7ffd93756d0965f9e1bc050b43adb38dfb74aca7c8f75944db"}
Nov 03 23:08:09.771704 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.770606    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj" event={"ID":"1538d524-6411-47f9-a8b6-fa97d897873a","Type":"ContainerStarted","Data":"2f66d5f05f9b5317217ca8d44a56b53e4325a46e0d621df7920c38231192e416"}
Nov 03 23:08:09.771704 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.770656    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj" event={"ID":"1538d524-6411-47f9-a8b6-fa97d897873a","Type":"ContainerStarted","Data":"f66d171ab826c78210b7d50e6371d4eb939ebdfc7e570f26ec34b084d3fadd1c"}
Nov 03 23:08:09.774100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.773836    5647 generic.go:356] "Generic (PLEG): container finished" podID="a599efd3-eee5-41ff-9172-93bc1f287003" containerID="e5bd53059e60c8c3bfac9a0a378e25f4aa9af00935d554ebfb1fe485645c464b" exitCode=0
Nov 03 23:08:09.774100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.773937    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" event={"ID":"a599efd3-eee5-41ff-9172-93bc1f287003","Type":"ContainerDied","Data":"e5bd53059e60c8c3bfac9a0a378e25f4aa9af00935d554ebfb1fe485645c464b"}
Nov 03 23:08:09.776327 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.776303    5647 generic.go:356] "Generic (PLEG): container finished" podID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerID="832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4" exitCode=0
Nov 03 23:08:09.776901 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.776369    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" event={"ID":"12957ee7-e8d8-4e4d-8060-6cf8fc91c410","Type":"ContainerDied","Data":"832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4"}
Nov 03 23:08:09.776901 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.776406    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj" event={"ID":"12957ee7-e8d8-4e4d-8060-6cf8fc91c410","Type":"ContainerDied","Data":"90b6802072e392d6f5c8c0c014e272c0bcaa00257cc4e1ddc6fedd4f0577627a"}
Nov 03 23:08:09.776901 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.776476    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"
Nov 03 23:08:09.789889 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.789380    5647 scope.go:122] "RemoveContainer" containerID="f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8"
Nov 03 23:08:09.790390 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:09.790344    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8\": container with ID starting with f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8 not found: ID does not exist" containerID="f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8"
Nov 03 23:08:09.790506 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.790394    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8"} err="failed to get container status \"f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8\": rpc error: code = NotFound desc = could not find container \"f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8\": container with ID starting with f2ef491b19a98d23d4a066ce3a6a0e8f2ecc234bd252ed3be6bd8d9c0a9a34e8 not found: ID does not exist"
Nov 03 23:08:09.790506 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.790419    5647 scope.go:122] "RemoveContainer" containerID="832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4"
Nov 03 23:08:09.820560 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.820538    5647 scope.go:122] "RemoveContainer" containerID="832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4"
Nov 03 23:08:09.820864 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:09.820842    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4\": container with ID starting with 832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4 not found: ID does not exist" containerID="832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4"
Nov 03 23:08:09.820961 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.820873    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4"} err="failed to get container status \"832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4\": rpc error: code = NotFound desc = could not find container \"832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4\": container with ID starting with 832130f3899f67fa0ac9e50c8f7f914692c5e6ec2fcda4905216516d9d72c1f4 not found: ID does not exist"
Nov 03 23:08:09.824209 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.824188    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 23:08:09.827330 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.827311    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 23:08:09.832502 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832485    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-podinfo\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.832570 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832544    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-credential-socket\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.832624 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832613    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-kube-api-access-p2qq2\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-kube-api-access-p2qq2\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.832708 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832663    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-socket\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.832767 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832716    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-token\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-token\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.832767 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832744    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81d99b28-c0aa-4312-9cd9-49fc570a7104-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81d99b28-c0aa-4312-9cd9-49fc570a7104-istiod-ca-cert\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.832868 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832788    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-envoy\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.832924 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832862    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-data\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.832924 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832904    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-certs\") pod \"81d99b28-c0aa-4312-9cd9-49fc570a7104\" (UID: \"81d99b28-c0aa-4312-9cd9-49fc570a7104\") "
Nov 03 23:08:09.833169 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832899    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-credential-socket" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.833394 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.832957    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-socket" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.833394 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833117    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-data" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.833642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833463    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-certs" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.833642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833136    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81d99b28-c0aa-4312-9cd9-49fc570a7104-istiod-ca-cert" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:09.833804 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833754    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.833804 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833780    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.833804 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833795    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.833980 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833809    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.833980 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833828    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.833980 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833841    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drb84\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-kube-api-access-drb84\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.833980 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833850    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.833980 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833858    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.833980 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.833867    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12957ee7-e8d8-4e4d-8060-6cf8fc91c410-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.834934 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.834912    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-kube-api-access-p2qq2" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "kube-api-access-p2qq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:09.835233 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.835212    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-token" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:09.835355 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.835336    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-podinfo" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:09.835430 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.835415    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-envoy" pod "81d99b28-c0aa-4312-9cd9-49fc570a7104" (UID: "81d99b28-c0aa-4312-9cd9-49fc570a7104"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.891474 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.891422    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj" podStartSLOduration=1.8914076149999999 podStartE2EDuration="1.891407615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:08:09.889846756 +0000 UTC m=+2713.330384016" watchObservedRunningTime="2026-11-03 23:08:09.891407615 +0000 UTC m=+2713.331944874"
Nov 03 23:08:09.934345 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934308    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-socket\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.934509 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934365    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a599efd3-eee5-41ff-9172-93bc1f287003-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a599efd3-eee5-41ff-9172-93bc1f287003-istio-podinfo\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.934509 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934395    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-envoy\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.934509 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934464    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-certs\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.934726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934593    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-envoy\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.934726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934634    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-credential-socket\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.934726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934641    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-socket" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.934726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934699    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-socket\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.934946 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934733    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-certs\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.934946 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934742    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-certs" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.934946 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934766    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-token\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-token\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.934946 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934795    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-data\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.934946 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934821    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66f55b8f-0627-455d-a540-3e02a6d25c01-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f55b8f-0627-455d-a540-3e02a6d25c01-istiod-ca-cert\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.934946 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934883    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-kube-api-access-fptpk\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-kube-api-access-fptpk\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.934946 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934911    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-istio-token\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-istio-token\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.934946 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934944    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-data\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934951    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-credential-socket" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934960    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-socket" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.934979    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-podinfo\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.935184    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-certs" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.935355    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-data" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.935373    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-data" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.935403    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-credential-socket\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.935452    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-kube-api-access-4zm9t\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-kube-api-access-4zm9t\") pod \"66f55b8f-0627-455d-a540-3e02a6d25c01\" (UID: \"66f55b8f-0627-455d-a540-3e02a6d25c01\") "
Nov 03 23:08:09.935762 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.935720    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a599efd3-eee5-41ff-9172-93bc1f287003-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a599efd3-eee5-41ff-9172-93bc1f287003-istiod-ca-cert\") pod \"a599efd3-eee5-41ff-9172-93bc1f287003\" (UID: \"a599efd3-eee5-41ff-9172-93bc1f287003\") "
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.935893    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-credential-socket" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.935922    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f55b8f-0627-455d-a540-3e02a6d25c01-istiod-ca-cert" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936353    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81d99b28-c0aa-4312-9cd9-49fc570a7104-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936374    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936408    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936423    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81d99b28-c0aa-4312-9cd9-49fc570a7104-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936437    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936451    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936456    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a599efd3-eee5-41ff-9172-93bc1f287003-istiod-ca-cert" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936466    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936500    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936520    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936537    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2qq2\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-kube-api-access-p2qq2\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936552    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936567    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936581    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f55b8f-0627-455d-a540-3e02a6d25c01-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936601    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81d99b28-c0aa-4312-9cd9-49fc570a7104-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.937218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.936615    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:09.938401 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.937387    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-podinfo" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:09.938401 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.937742    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a599efd3-eee5-41ff-9172-93bc1f287003-istio-podinfo" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:09.938401 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.937959    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-token" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:09.938401 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.938089    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-envoy" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:09.938401 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.938086    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-istio-token" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:09.938701 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.938529    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-kube-api-access-fptpk" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "kube-api-access-fptpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:09.939039 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.939013    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-kube-api-access-4zm9t" pod "66f55b8f-0627-455d-a540-3e02a6d25c01" (UID: "66f55b8f-0627-455d-a540-3e02a6d25c01"). InnerVolumeSpecName "kube-api-access-4zm9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:09.939116 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:09.939097    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-envoy" pod "a599efd3-eee5-41ff-9172-93bc1f287003" (UID: "a599efd3-eee5-41ff-9172-93bc1f287003"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:10.038285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038196    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a599efd3-eee5-41ff-9172-93bc1f287003-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.038285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038230    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a599efd3-eee5-41ff-9172-93bc1f287003-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.038285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038240    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.038285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038248    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.038285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038257    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fptpk\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-kube-api-access-fptpk\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.038285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038266    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a599efd3-eee5-41ff-9172-93bc1f287003-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.038285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038274    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f55b8f-0627-455d-a540-3e02a6d25c01-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.038285 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038285    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zm9t\" (UniqueName: \"kubernetes.io/projected/66f55b8f-0627-455d-a540-3e02a6d25c01-kube-api-access-4zm9t\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.038663 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.038293    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a599efd3-eee5-41ff-9172-93bc1f287003-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:10.121793 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.121627    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"]
Nov 03 23:08:10.126028 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.125998    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj"]
Nov 03 23:08:10.473032 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.472999    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"]
Nov 03 23:08:10.477199 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.477173    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg"]
Nov 03 23:08:10.695515 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.695483    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" path="/var/lib/kubelet/pods/12957ee7-e8d8-4e4d-8060-6cf8fc91c410/volumes"
Nov 03 23:08:10.695947 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.695932    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" path="/var/lib/kubelet/pods/81d99b28-c0aa-4312-9cd9-49fc570a7104/volumes"
Nov 03 23:08:10.783020 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.782987    5647 generic.go:356] "Generic (PLEG): container finished" podID="1538d524-6411-47f9-a8b6-fa97d897873a" containerID="2f66d5f05f9b5317217ca8d44a56b53e4325a46e0d621df7920c38231192e416" exitCode=0
Nov 03 23:08:10.783263 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.783027    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj" event={"ID":"1538d524-6411-47f9-a8b6-fa97d897873a","Type":"ContainerDied","Data":"2f66d5f05f9b5317217ca8d44a56b53e4325a46e0d621df7920c38231192e416"}
Nov 03 23:08:10.784405 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.784388    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"
Nov 03 23:08:10.784533 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.784416    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp" event={"ID":"a599efd3-eee5-41ff-9172-93bc1f287003","Type":"ContainerDied","Data":"606ecb16a9b496971975362819f487ccf46045c450e02b22608dda3fe251bc8e"}
Nov 03 23:08:10.784533 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.784462    5647 scope.go:122] "RemoveContainer" containerID="e5bd53059e60c8c3bfac9a0a378e25f4aa9af00935d554ebfb1fe485645c464b"
Nov 03 23:08:10.788820 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.788797    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c" event={"ID":"66f55b8f-0627-455d-a540-3e02a6d25c01","Type":"ContainerDied","Data":"975ecf23617e74e5a1b93fa848aea03941e2f791a5c174dc1f330051c0ea816d"}
Nov 03 23:08:10.788957 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.788821    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"
Nov 03 23:08:10.795592 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:10.795420    5647 scope.go:122] "RemoveContainer" containerID="92432074c9316d7ffd93756d0965f9e1bc050b43adb38dfb74aca7c8f75944db"
Nov 03 23:08:11.011971 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.011871    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"]
Nov 03 23:08:11.016288 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.016264    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c"]
Nov 03 23:08:11.017371 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.017338    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main" containerID="cri-o://902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c" gracePeriod=45
Nov 03 23:08:11.125731 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.125697    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"]
Nov 03 23:08:11.129993 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.129963    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp"]
Nov 03 23:08:11.273623 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.273558    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 23:08:11.352491 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.352456    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-tmp-dir\") pod \"a38135ba-2abe-4853-91a4-f4982fb3f727\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") "
Nov 03 23:08:11.352658 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.352569    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a38135ba-2abe-4853-91a4-f4982fb3f727-kube-api-access-v5kld\" (UniqueName: \"kubernetes.io/projected/a38135ba-2abe-4853-91a4-f4982fb3f727-kube-api-access-v5kld\") pod \"a38135ba-2abe-4853-91a4-f4982fb3f727\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") "
Nov 03 23:08:11.352658 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.352644    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-model-cache\") pod \"a38135ba-2abe-4853-91a4-f4982fb3f727\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") "
Nov 03 23:08:11.352837 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.352812    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-tmp-dir" pod "a38135ba-2abe-4853-91a4-f4982fb3f727" (UID: "a38135ba-2abe-4853-91a4-f4982fb3f727"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:11.352917 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.352897    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-model-cache" pod "a38135ba-2abe-4853-91a4-f4982fb3f727" (UID: "a38135ba-2abe-4853-91a4-f4982fb3f727"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:11.353078 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.353062    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:11.353118 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.353085    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:11.355112 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.355094    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38135ba-2abe-4853-91a4-f4982fb3f727-kube-api-access-v5kld" pod "a38135ba-2abe-4853-91a4-f4982fb3f727" (UID: "a38135ba-2abe-4853-91a4-f4982fb3f727"). InnerVolumeSpecName "kube-api-access-v5kld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:11.453944 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.453915    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a38135ba-2abe-4853-91a4-f4982fb3f727-tls-certs\" (UniqueName: \"kubernetes.io/secret/a38135ba-2abe-4853-91a4-f4982fb3f727-tls-certs\") pod \"a38135ba-2abe-4853-91a4-f4982fb3f727\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") "
Nov 03 23:08:11.454137 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.454099    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-home\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-home\") pod \"a38135ba-2abe-4853-91a4-f4982fb3f727\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") "
Nov 03 23:08:11.454208 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.454185    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-dshm\") pod \"a38135ba-2abe-4853-91a4-f4982fb3f727\" (UID: \"a38135ba-2abe-4853-91a4-f4982fb3f727\") "
Nov 03 23:08:11.454384 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.454361    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-home" pod "a38135ba-2abe-4853-91a4-f4982fb3f727" (UID: "a38135ba-2abe-4853-91a4-f4982fb3f727"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:11.454570 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.454548    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:11.454570 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.454567    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5kld\" (UniqueName: \"kubernetes.io/projected/a38135ba-2abe-4853-91a4-f4982fb3f727-kube-api-access-v5kld\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:11.456509 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.456481    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a38135ba-2abe-4853-91a4-f4982fb3f727-tls-certs" pod "a38135ba-2abe-4853-91a4-f4982fb3f727" (UID: "a38135ba-2abe-4853-91a4-f4982fb3f727"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:08:11.456639 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.456584    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-dshm" pod "a38135ba-2abe-4853-91a4-f4982fb3f727" (UID: "a38135ba-2abe-4853-91a4-f4982fb3f727"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:11.555691 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.555602    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38135ba-2abe-4853-91a4-f4982fb3f727-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:11.555691 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.555632    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38135ba-2abe-4853-91a4-f4982fb3f727-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:11.795771 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.795734    5647 generic.go:356] "Generic (PLEG): container finished" podID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerID="902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c" exitCode=0
Nov 03 23:08:11.796172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.795781    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"
Nov 03 23:08:11.796172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.795820    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" event={"ID":"a38135ba-2abe-4853-91a4-f4982fb3f727","Type":"ContainerDied","Data":"902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c"}
Nov 03 23:08:11.796172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.795866    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s" event={"ID":"a38135ba-2abe-4853-91a4-f4982fb3f727","Type":"ContainerDied","Data":"e1d6f403b4e028cec949aba6df9604b7906f2326b5794c06dc0c5cfc87322356"}
Nov 03 23:08:11.796172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.795885    5647 scope.go:122] "RemoveContainer" containerID="902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c"
Nov 03 23:08:11.805567 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.805546    5647 scope.go:122] "RemoveContainer" containerID="902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c"
Nov 03 23:08:11.805914 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:11.805861    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c\": container with ID starting with 902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c not found: ID does not exist" containerID="902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c"
Nov 03 23:08:11.805914 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.805895    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c"} err="failed to get container status \"902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c\": rpc error: code = NotFound desc = could not find container \"902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c\": container with ID starting with 902bcc28db10db6d5df7aa327abee03875929c4722948c9d170e57e06ce2987c not found: ID does not exist"
Nov 03 23:08:11.940432 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.940400    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:11.965478 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.965450    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"]
Nov 03 23:08:11.971983 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:11.971962    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-1d31f8/simulator-w3-1d31f8-kserve-665d69d48c-phq7s"]
Nov 03 23:08:12.061273 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.061194    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-xks-ca\") pod \"1538d524-6411-47f9-a8b6-fa97d897873a\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") "
Nov 03 23:08:12.061430 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.061294    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-openshift-ca\") pod \"1538d524-6411-47f9-a8b6-fa97d897873a\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") "
Nov 03 23:08:12.061430 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.061331    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1538d524-6411-47f9-a8b6-fa97d897873a-kube-api-access-gbq9r\" (UniqueName: \"kubernetes.io/projected/1538d524-6411-47f9-a8b6-fa97d897873a-kube-api-access-gbq9r\") pod \"1538d524-6411-47f9-a8b6-fa97d897873a\" (UID: \"1538d524-6411-47f9-a8b6-fa97d897873a\") "
Nov 03 23:08:12.061773 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.061742    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-openshift-ca" pod "1538d524-6411-47f9-a8b6-fa97d897873a" (UID: "1538d524-6411-47f9-a8b6-fa97d897873a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:12.063556 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.063526    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-xks-ca" pod "1538d524-6411-47f9-a8b6-fa97d897873a" (UID: "1538d524-6411-47f9-a8b6-fa97d897873a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:08:12.063837 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.063813    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1538d524-6411-47f9-a8b6-fa97d897873a-kube-api-access-gbq9r" pod "1538d524-6411-47f9-a8b6-fa97d897873a" (UID: "1538d524-6411-47f9-a8b6-fa97d897873a"). InnerVolumeSpecName "kube-api-access-gbq9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:12.162507 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.162472    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-xks-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:12.162507 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.162501    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1538d524-6411-47f9-a8b6-fa97d897873a-maas-api-openshift-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:12.162507 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.162511    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbq9r\" (UniqueName: \"kubernetes.io/projected/1538d524-6411-47f9-a8b6-fa97d897873a-kube-api-access-gbq9r\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:12.696752 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.696717    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" path="/var/lib/kubelet/pods/66f55b8f-0627-455d-a540-3e02a6d25c01/volumes"
Nov 03 23:08:12.697132 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.697118    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" path="/var/lib/kubelet/pods/a38135ba-2abe-4853-91a4-f4982fb3f727/volumes"
Nov 03 23:08:12.697502 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.697487    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" path="/var/lib/kubelet/pods/a599efd3-eee5-41ff-9172-93bc1f287003/volumes"
Nov 03 23:08:12.805858 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.805828    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"
Nov 03 23:08:12.806264 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.805827    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj" event={"ID":"1538d524-6411-47f9-a8b6-fa97d897873a","Type":"ContainerDied","Data":"f66d171ab826c78210b7d50e6371d4eb939ebdfc7e570f26ec34b084d3fadd1c"}
Nov 03 23:08:12.806264 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:12.805955    5647 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f66d171ab826c78210b7d50e6371d4eb939ebdfc7e570f26ec34b084d3fadd1c"
Nov 03 23:08:13.392180 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.392146    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"]
Nov 03 23:08:13.471613 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.471575    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"]
Nov 03 23:08:13.842576 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.842543    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"]
Nov 03 23:08:13.843066 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843049    5647 cpu_manager.go:414] "containerMap: removing container" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy"
Nov 03 23:08:13.843113 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843067    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy"
Nov 03 23:08:13.843113 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843092    5647 cpu_manager.go:414] "containerMap: removing container" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy"
Nov 03 23:08:13.843113 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843097    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d206c63-b944-4b5b-8952-102b185b6925" containerName="istio-proxy"
Nov 03 23:08:13.843113 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843109    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843116    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843128    5647 cpu_manager.go:414] "containerMap: removing container" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843136    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843151    5647 cpu_manager.go:414] "containerMap: removing container" podUID="1538d524-6411-47f9-a8b6-fa97d897873a" containerName="revoke-keys"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843159    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1538d524-6411-47f9-a8b6-fa97d897873a" containerName="revoke-keys"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843177    5647 cpu_manager.go:414] "containerMap: removing container" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843185    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843218    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main"
Nov 03 23:08:13.843232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843226    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main"
Nov 03 23:08:13.843494 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843325    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="1538d524-6411-47f9-a8b6-fa97d897873a" containerName="revoke-keys"
Nov 03 23:08:13.843494 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843335    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="81d99b28-c0aa-4312-9cd9-49fc570a7104" containerName="istio-proxy"
Nov 03 23:08:13.843494 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843343    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a38135ba-2abe-4853-91a4-f4982fb3f727" containerName="main"
Nov 03 23:08:13.843494 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843349    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea7424a3-4b15-44af-beae-c41b3eb13801" containerName="istio-proxy"
Nov 03 23:08:13.843494 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843355    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="12957ee7-e8d8-4e4d-8060-6cf8fc91c410" containerName="istio-proxy"
Nov 03 23:08:13.843494 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843365    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="66f55b8f-0627-455d-a540-3e02a6d25c01" containerName="istio-proxy"
Nov 03 23:08:13.843494 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.843370    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a599efd3-eee5-41ff-9172-93bc1f287003" containerName="istio-proxy"
Nov 03 23:08:13.846815 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.846796    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:13.908519 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.908487    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"]
Nov 03 23:08:13.976014 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.975967    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 23:08:13.976172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.976086    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p2s2g\""
Nov 03 23:08:13.981594 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.981574    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:13.981808 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.981786    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:13.981876 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:13.981839    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr7tw\" (UniqueName: \"kubernetes.io/projected/70bb69e8-6434-42fd-b5cc-b1236f463311-kube-api-access-nr7tw\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:14.083123 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.083077    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:14.083356 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.083133    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr7tw\" (UniqueName: \"kubernetes.io/projected/70bb69e8-6434-42fd-b5cc-b1236f463311-kube-api-access-nr7tw\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:14.083356 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.083328    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:14.083837 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.083817    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:14.085966 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.085945    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:14.143402 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.143329    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr7tw\" (UniqueName: \"kubernetes.io/projected/70bb69e8-6434-42fd-b5cc-b1236f463311-kube-api-access-nr7tw\") pod \"maas-api-revoke-sub-61b6d326-q6q2m\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:14.156405 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.156372    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"]
Nov 03 23:08:14.159866 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.159844    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.160821 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.160799    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:14.219109 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.218232    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"]
Nov 03 23:08:14.285474 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.285444    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6hv7\" (UniqueName: \"kubernetes.io/projected/a8e91d27-a1af-47df-8369-346bf5b8b7bc-kube-api-access-s6hv7\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.285635 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.285484    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.285635 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.285612    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.387245 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.387207    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hv7\" (UniqueName: \"kubernetes.io/projected/a8e91d27-a1af-47df-8369-346bf5b8b7bc-kube-api-access-s6hv7\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.387440 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.387253    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.387440 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.387313    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.388051 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.388030    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.389879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.389860    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.447864 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.447791    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6hv7\" (UniqueName: \"kubernetes.io/projected/a8e91d27-a1af-47df-8369-346bf5b8b7bc-kube-api-access-s6hv7\") pod \"maas-api-revoke-sub-4288120c-cvn6p\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.476626 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.476596    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:14.545300 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.545270    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"]
Nov 03 23:08:14.549358 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:08:14.549324    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70bb69e8_6434_42fd_b5cc_b1236f463311.slice/crio-1bc454d5ec0328ec2ae3aa58e0f19e7335ca789ef2f0983cd51487f7dc88371f WatchSource:0}: Error finding container 1bc454d5ec0328ec2ae3aa58e0f19e7335ca789ef2f0983cd51487f7dc88371f: Status 404 returned error can't find the container with id 1bc454d5ec0328ec2ae3aa58e0f19e7335ca789ef2f0983cd51487f7dc88371f
Nov 03 23:08:14.819706 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.819656    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m" event={"ID":"70bb69e8-6434-42fd-b5cc-b1236f463311","Type":"ContainerStarted","Data":"1bc454d5ec0328ec2ae3aa58e0f19e7335ca789ef2f0983cd51487f7dc88371f"}
Nov 03 23:08:14.850378 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:14.850351    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"]
Nov 03 23:08:14.852768 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:08:14.852740    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e91d27_a1af_47df_8369_346bf5b8b7bc.slice/crio-ebbbc4e94db2d08e845cf085350a793169932229dfaecfbe418df2d010e790c4 WatchSource:0}: Error finding container ebbbc4e94db2d08e845cf085350a793169932229dfaecfbe418df2d010e790c4: Status 404 returned error can't find the container with id ebbbc4e94db2d08e845cf085350a793169932229dfaecfbe418df2d010e790c4
Nov 03 23:08:15.295347 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:15.295312    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:08:15.826317 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:15.826281    5647 generic.go:356] "Generic (PLEG): container finished" podID="a8e91d27-a1af-47df-8369-346bf5b8b7bc" containerID="a2e3829b392b929aa9580dc8cfe423b44c037e872ca14650fccc27285b52d2f2" exitCode=0
Nov 03 23:08:15.826519 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:15.826367    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p" event={"ID":"a8e91d27-a1af-47df-8369-346bf5b8b7bc","Type":"ContainerDied","Data":"a2e3829b392b929aa9580dc8cfe423b44c037e872ca14650fccc27285b52d2f2"}
Nov 03 23:08:15.826519 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:15.826418    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p" event={"ID":"a8e91d27-a1af-47df-8369-346bf5b8b7bc","Type":"ContainerStarted","Data":"ebbbc4e94db2d08e845cf085350a793169932229dfaecfbe418df2d010e790c4"}
Nov 03 23:08:15.827998 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:15.827976    5647 generic.go:356] "Generic (PLEG): container finished" podID="70bb69e8-6434-42fd-b5cc-b1236f463311" containerID="69bb53e29bdcc8b6128b0c3a55ee8333d2530aae5105c9d742ac81178d78eac8" exitCode=0
Nov 03 23:08:15.828101 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:15.828053    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m" event={"ID":"70bb69e8-6434-42fd-b5cc-b1236f463311","Type":"ContainerDied","Data":"69bb53e29bdcc8b6128b0c3a55ee8333d2530aae5105c9d742ac81178d78eac8"}
Nov 03 23:08:16.042748 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:16.042639    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:08:16.973360 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:16.973328    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:17.004527 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.004499    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:17.113826 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.113736    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8e91d27-a1af-47df-8369-346bf5b8b7bc-kube-api-access-s6hv7\" (UniqueName: \"kubernetes.io/projected/a8e91d27-a1af-47df-8369-346bf5b8b7bc-kube-api-access-s6hv7\") pod \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") "
Nov 03 23:08:17.114287 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.113830    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70bb69e8-6434-42fd-b5cc-b1236f463311-kube-api-access-nr7tw\" (UniqueName: \"kubernetes.io/projected/70bb69e8-6434-42fd-b5cc-b1236f463311-kube-api-access-nr7tw\") pod \"70bb69e8-6434-42fd-b5cc-b1236f463311\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") "
Nov 03 23:08:17.114287 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.113895    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-openshift-ca\") pod \"70bb69e8-6434-42fd-b5cc-b1236f463311\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") "
Nov 03 23:08:17.114287 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.113954    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-xks-ca\") pod \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") "
Nov 03 23:08:17.114287 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.114056    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-openshift-ca\") pod \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\" (UID: \"a8e91d27-a1af-47df-8369-346bf5b8b7bc\") "
Nov 03 23:08:17.114287 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.114094    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-xks-ca\") pod \"70bb69e8-6434-42fd-b5cc-b1236f463311\" (UID: \"70bb69e8-6434-42fd-b5cc-b1236f463311\") "
Nov 03 23:08:17.114606 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.114284    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-openshift-ca" pod "70bb69e8-6434-42fd-b5cc-b1236f463311" (UID: "70bb69e8-6434-42fd-b5cc-b1236f463311"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:17.114702 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.114574    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-openshift-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:17.115019 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.114961    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-openshift-ca" pod "a8e91d27-a1af-47df-8369-346bf5b8b7bc" (UID: "a8e91d27-a1af-47df-8369-346bf5b8b7bc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:17.116294 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.116262    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8e91d27-a1af-47df-8369-346bf5b8b7bc-kube-api-access-s6hv7" pod "a8e91d27-a1af-47df-8369-346bf5b8b7bc" (UID: "a8e91d27-a1af-47df-8369-346bf5b8b7bc"). InnerVolumeSpecName "kube-api-access-s6hv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:17.116412 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.116349    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-xks-ca" pod "70bb69e8-6434-42fd-b5cc-b1236f463311" (UID: "70bb69e8-6434-42fd-b5cc-b1236f463311"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:08:17.116886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.116869    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70bb69e8-6434-42fd-b5cc-b1236f463311-kube-api-access-nr7tw" pod "70bb69e8-6434-42fd-b5cc-b1236f463311" (UID: "70bb69e8-6434-42fd-b5cc-b1236f463311"). InnerVolumeSpecName "kube-api-access-nr7tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:17.117050 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.117032    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-xks-ca" pod "a8e91d27-a1af-47df-8369-346bf5b8b7bc" (UID: "a8e91d27-a1af-47df-8369-346bf5b8b7bc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:08:17.215461 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.215426    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-openshift-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:17.215461 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.215456    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/70bb69e8-6434-42fd-b5cc-b1236f463311-maas-api-xks-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:17.215461 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.215467    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6hv7\" (UniqueName: \"kubernetes.io/projected/a8e91d27-a1af-47df-8369-346bf5b8b7bc-kube-api-access-s6hv7\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:17.215718 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.215478    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr7tw\" (UniqueName: \"kubernetes.io/projected/70bb69e8-6434-42fd-b5cc-b1236f463311-kube-api-access-nr7tw\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:17.215718 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.215486    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8e91d27-a1af-47df-8369-346bf5b8b7bc-maas-api-xks-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:17.839404 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.839370    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"
Nov 03 23:08:17.839404 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.839381    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p" event={"ID":"a8e91d27-a1af-47df-8369-346bf5b8b7bc","Type":"ContainerDied","Data":"ebbbc4e94db2d08e845cf085350a793169932229dfaecfbe418df2d010e790c4"}
Nov 03 23:08:17.839726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.839436    5647 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebbbc4e94db2d08e845cf085350a793169932229dfaecfbe418df2d010e790c4"
Nov 03 23:08:17.840797 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.840777    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"
Nov 03 23:08:17.840922 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.840817    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m" event={"ID":"70bb69e8-6434-42fd-b5cc-b1236f463311","Type":"ContainerDied","Data":"1bc454d5ec0328ec2ae3aa58e0f19e7335ca789ef2f0983cd51487f7dc88371f"}
Nov 03 23:08:17.840922 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:17.840849    5647 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bc454d5ec0328ec2ae3aa58e0f19e7335ca789ef2f0983cd51487f7dc88371f"
Nov 03 23:08:18.392724 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:18.392624    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h" podUID="d0fb062c-7b7b-42c1-a99d-978d35c02be9" containerName="payload-processing" containerID="cri-o://89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133" gracePeriod=25
Nov 03 23:08:18.472372 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:18.472285    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf" podUID="dfd62750-e701-4dd2-8693-752677fafdd4" containerName="payload-pre-processing" containerID="cri-o://d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b" gracePeriod=25
Nov 03 23:08:20.787287 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:20.787244    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h" podUID="d0fb062c-7b7b-42c1-a99d-978d35c02be9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 23:08:20.787724 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:20.787290    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf" podUID="dfd62750-e701-4dd2-8693-752677fafdd4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 23:08:21.234372 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.234323    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" containerID="cri-o://f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6" gracePeriod=45
Nov 03 23:08:21.498162 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.498086    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 23:08:21.557464 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557425    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tmp-dir\") pod \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") "
Nov 03 23:08:21.557464 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557467    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-home\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-home\") pod \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") "
Nov 03 23:08:21.557786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557544    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-model-cache\") pod \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") "
Nov 03 23:08:21.557786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557580    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-dshm\") pod \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") "
Nov 03 23:08:21.557786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557626    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tls-certs\" (UniqueName: \"kubernetes.io/secret/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tls-certs\") pod \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") "
Nov 03 23:08:21.557786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557699    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-kube-api-access-mvqpv\" (UniqueName: \"kubernetes.io/projected/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-kube-api-access-mvqpv\") pod \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\" (UID: \"0315c1df-36d9-41a1-a3f9-581bb5aee4dd\") "
Nov 03 23:08:21.557786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557758    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tmp-dir" pod "0315c1df-36d9-41a1-a3f9-581bb5aee4dd" (UID: "0315c1df-36d9-41a1-a3f9-581bb5aee4dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.558042 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557802    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-model-cache" pod "0315c1df-36d9-41a1-a3f9-581bb5aee4dd" (UID: "0315c1df-36d9-41a1-a3f9-581bb5aee4dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.558042 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.557834    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-home" pod "0315c1df-36d9-41a1-a3f9-581bb5aee4dd" (UID: "0315c1df-36d9-41a1-a3f9-581bb5aee4dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.558147 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.558107    5647 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tmp-dir\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.558147 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.558125    5647 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-home\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.558147 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.558138    5647 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-model-cache\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.560163 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.560139    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-dshm" pod "0315c1df-36d9-41a1-a3f9-581bb5aee4dd" (UID: "0315c1df-36d9-41a1-a3f9-581bb5aee4dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:21.560163 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.560153    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tls-certs" pod "0315c1df-36d9-41a1-a3f9-581bb5aee4dd" (UID: "0315c1df-36d9-41a1-a3f9-581bb5aee4dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:08:21.560301 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.560188    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-kube-api-access-mvqpv" pod "0315c1df-36d9-41a1-a3f9-581bb5aee4dd" (UID: "0315c1df-36d9-41a1-a3f9-581bb5aee4dd"). InnerVolumeSpecName "kube-api-access-mvqpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:21.659740 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.659708    5647 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-dshm\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.659740 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.659736    5647 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-tls-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.659740 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.659746    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvqpv\" (UniqueName: \"kubernetes.io/projected/0315c1df-36d9-41a1-a3f9-581bb5aee4dd-kube-api-access-mvqpv\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:21.864941 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.864848    5647 generic.go:356] "Generic (PLEG): container finished" podID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerID="f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6" exitCode=0
Nov 03 23:08:21.864941 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.864929    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"
Nov 03 23:08:21.865540 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.864932    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" event={"ID":"0315c1df-36d9-41a1-a3f9-581bb5aee4dd","Type":"ContainerDied","Data":"f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6"}
Nov 03 23:08:21.865540 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.865049    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" event={"ID":"0315c1df-36d9-41a1-a3f9-581bb5aee4dd","Type":"ContainerDied","Data":"cefd7bc6adcc83535f60f2a740a99be54c76aa33017bac6d372015b6c4abb423"}
Nov 03 23:08:21.865540 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.865071    5647 scope.go:122] "RemoveContainer" containerID="f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6"
Nov 03 23:08:21.875464 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.875289    5647 scope.go:122] "RemoveContainer" containerID="f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6"
Nov 03 23:08:21.875607 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:21.875583    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6\": container with ID starting with f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6 not found: ID does not exist" containerID="f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6"
Nov 03 23:08:21.875726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:21.875612    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6"} err="failed to get container status \"f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6\": rpc error: code = NotFound desc = could not find container \"f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6\": container with ID starting with f5abc32069608f4f986e3d3e3490e445b8ddeed989f73cea1c6f8da62e1276e6 not found: ID does not exist"
Nov 03 23:08:22.067690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:22.064659    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"]
Nov 03 23:08:22.074901 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:22.074875    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5"]
Nov 03 23:08:22.433086 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:22.433047    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-32449d/distinct-2-w4-32449d-kserve-dc7b589c4-hn8t5" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.69:8000/ready\": context deadline exceeded"
Nov 03 23:08:22.694895 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:22.694809    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" path="/var/lib/kubelet/pods/0315c1df-36d9-41a1-a3f9-581bb5aee4dd/volumes"
Nov 03 23:08:30.787780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:30.787735    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf" podUID="dfd62750-e701-4dd2-8693-752677fafdd4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 23:08:30.790204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:30.787752    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h" podUID="d0fb062c-7b7b-42c1-a99d-978d35c02be9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 23:08:36.035936 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:36.035900    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"]
Nov 03 23:08:36.036348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:36.036141    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" containerID="cri-o://6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2" gracePeriod=30
Nov 03 23:08:38.731106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.731080    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 23:08:38.748840 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.748816    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 23:08:38.854158 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.854129    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfd62750-e701-4dd2-8693-752677fafdd4-kube-api-access-njxhd\" (UniqueName: \"kubernetes.io/projected/dfd62750-e701-4dd2-8693-752677fafdd4-kube-api-access-njxhd\") pod \"dfd62750-e701-4dd2-8693-752677fafdd4\" (UID: \"dfd62750-e701-4dd2-8693-752677fafdd4\") "
Nov 03 23:08:38.854325 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.854275    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0fb062c-7b7b-42c1-a99d-978d35c02be9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0fb062c-7b7b-42c1-a99d-978d35c02be9-plugins-config-volume\") pod \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\" (UID: \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\") "
Nov 03 23:08:38.854325 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.854316    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0fb062c-7b7b-42c1-a99d-978d35c02be9-kube-api-access-fxq6g\" (UniqueName: \"kubernetes.io/projected/d0fb062c-7b7b-42c1-a99d-978d35c02be9-kube-api-access-fxq6g\") pod \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\" (UID: \"d0fb062c-7b7b-42c1-a99d-978d35c02be9\") "
Nov 03 23:08:38.854447 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.854425    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dfd62750-e701-4dd2-8693-752677fafdd4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfd62750-e701-4dd2-8693-752677fafdd4-plugins-config-volume\") pod \"dfd62750-e701-4dd2-8693-752677fafdd4\" (UID: \"dfd62750-e701-4dd2-8693-752677fafdd4\") "
Nov 03 23:08:38.854717 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.854687    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0fb062c-7b7b-42c1-a99d-978d35c02be9-plugins-config-volume" pod "d0fb062c-7b7b-42c1-a99d-978d35c02be9" (UID: "d0fb062c-7b7b-42c1-a99d-978d35c02be9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:38.854850 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.854724    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd62750-e701-4dd2-8693-752677fafdd4-plugins-config-volume" pod "dfd62750-e701-4dd2-8693-752677fafdd4" (UID: "dfd62750-e701-4dd2-8693-752677fafdd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:38.855084 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.854849    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0fb062c-7b7b-42c1-a99d-978d35c02be9-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.855084 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.854866    5647 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfd62750-e701-4dd2-8693-752677fafdd4-plugins-config-volume\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.856500 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.856479    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd62750-e701-4dd2-8693-752677fafdd4-kube-api-access-njxhd" pod "dfd62750-e701-4dd2-8693-752677fafdd4" (UID: "dfd62750-e701-4dd2-8693-752677fafdd4"). InnerVolumeSpecName "kube-api-access-njxhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:38.856500 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.856495    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0fb062c-7b7b-42c1-a99d-978d35c02be9-kube-api-access-fxq6g" pod "d0fb062c-7b7b-42c1-a99d-978d35c02be9" (UID: "d0fb062c-7b7b-42c1-a99d-978d35c02be9"). InnerVolumeSpecName "kube-api-access-fxq6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:38.955456 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.955422    5647 generic.go:356] "Generic (PLEG): container finished" podID="d0fb062c-7b7b-42c1-a99d-978d35c02be9" containerID="89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133" exitCode=0
Nov 03 23:08:38.955618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.955466    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njxhd\" (UniqueName: \"kubernetes.io/projected/dfd62750-e701-4dd2-8693-752677fafdd4-kube-api-access-njxhd\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.955618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.955477    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"
Nov 03 23:08:38.955618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.955484    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxq6g\" (UniqueName: \"kubernetes.io/projected/d0fb062c-7b7b-42c1-a99d-978d35c02be9-kube-api-access-fxq6g\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:38.955618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.955512    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h" event={"ID":"d0fb062c-7b7b-42c1-a99d-978d35c02be9","Type":"ContainerDied","Data":"89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133"}
Nov 03 23:08:38.955618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.955570    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h" event={"ID":"d0fb062c-7b7b-42c1-a99d-978d35c02be9","Type":"ContainerDied","Data":"e2da178550f97f41a50d6583dbd15046b845e4664d17d8c7469d645ad145e235"}
Nov 03 23:08:38.955618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.955602    5647 scope.go:122] "RemoveContainer" containerID="89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133"
Nov 03 23:08:38.956980 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.956951    5647 generic.go:356] "Generic (PLEG): container finished" podID="dfd62750-e701-4dd2-8693-752677fafdd4" containerID="d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b" exitCode=0
Nov 03 23:08:38.957097 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.956988    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf" event={"ID":"dfd62750-e701-4dd2-8693-752677fafdd4","Type":"ContainerDied","Data":"d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b"}
Nov 03 23:08:38.957097 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.956999    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"
Nov 03 23:08:38.957097 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.957028    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf" event={"ID":"dfd62750-e701-4dd2-8693-752677fafdd4","Type":"ContainerDied","Data":"b027f5be63ff32ad017a7b9538e2945a65265b963042995ae54ee01a2a4a2cb6"}
Nov 03 23:08:38.967497 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.967415    5647 scope.go:122] "RemoveContainer" containerID="89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133"
Nov 03 23:08:38.967795 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:38.967772    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133\": container with ID starting with 89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133 not found: ID does not exist" containerID="89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133"
Nov 03 23:08:38.967891 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.967807    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133"} err="failed to get container status \"89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133\": rpc error: code = NotFound desc = could not find container \"89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133\": container with ID starting with 89ed5b817a9f704096da7bd07b9ced4eb7cd9ebed9669d84ea827de1767da133 not found: ID does not exist"
Nov 03 23:08:38.967891 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.967833    5647 scope.go:122] "RemoveContainer" containerID="d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b"
Nov 03 23:08:38.976143 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.976122    5647 scope.go:122] "RemoveContainer" containerID="d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b"
Nov 03 23:08:38.976401 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:38.976383    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b\": container with ID starting with d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b not found: ID does not exist" containerID="d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b"
Nov 03 23:08:38.976484 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:38.976407    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b"} err="failed to get container status \"d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b\": rpc error: code = NotFound desc = could not find container \"d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b\": container with ID starting with d55c4c6d6237a0c55e9d4ba6c11a3cbe882e6660f436860fa46efc960a17df1b not found: ID does not exist"
Nov 03 23:08:39.179846 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:39.179811    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"]
Nov 03 23:08:39.184622 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:39.184593    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf"]
Nov 03 23:08:39.292083 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:39.292003    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"]
Nov 03 23:08:39.297882 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:39.297847    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h"]
Nov 03 23:08:40.694474 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:40.694440    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0fb062c-7b7b-42c1-a99d-978d35c02be9" path="/var/lib/kubelet/pods/d0fb062c-7b7b-42c1-a99d-978d35c02be9/volumes"
Nov 03 23:08:40.694872 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:40.694803    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfd62750-e701-4dd2-8693-752677fafdd4" path="/var/lib/kubelet/pods/dfd62750-e701-4dd2-8693-752677fafdd4/volumes"
Nov 03 23:08:40.916238 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:40.916196    5647 patch_prober.go:29] interesting pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 23:08:40.916397 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:40.916261    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 23:08:41.289708 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.289661    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 23:08:41.380388 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380295    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-credential-socket\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.380388 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380342    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-podinfo\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.380388 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380372    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-socket\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.380649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380418    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-token\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-token\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.380649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380438    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-certs\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.380649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380476    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-envoy\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.380649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380547    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-kube-api-access-s2657\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-kube-api-access-s2657\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.380649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380592    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d8040b2-0353-449a-a8b6-01b6be112ad0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d8040b2-0353-449a-a8b6-01b6be112ad0-istiod-ca-cert\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.380649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380609    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-credential-socket" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:41.380649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380629    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-data\") pod \"0d8040b2-0353-449a-a8b6-01b6be112ad0\" (UID: \"0d8040b2-0353-449a-a8b6-01b6be112ad0\") "
Nov 03 23:08:41.381085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.380821    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-socket" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:41.381085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.381014    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.381085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.381033    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.381085 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.381033    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d8040b2-0353-449a-a8b6-01b6be112ad0-istiod-ca-cert" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:41.381306 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.381174    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-data" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:41.381360 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.381306    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-certs" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:41.382885 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.382865    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-podinfo" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:41.383087 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.383064    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-token" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:41.383374 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.383356    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-kube-api-access-s2657" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "kube-api-access-s2657". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:41.383374 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.383363    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-envoy" pod "0d8040b2-0353-449a-a8b6-01b6be112ad0" (UID: "0d8040b2-0353-449a-a8b6-01b6be112ad0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:41.482121 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.482085    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2657\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-kube-api-access-s2657\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.482121 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.482115    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d8040b2-0353-449a-a8b6-01b6be112ad0-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.482121 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.482125    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.482350 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.482133    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.482350 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.482142    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.482350 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.482150    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.482350 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.482158    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d8040b2-0353-449a-a8b6-01b6be112ad0-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:41.976817 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.976784    5647 generic.go:356] "Generic (PLEG): container finished" podID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerID="6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2" exitCode=0
Nov 03 23:08:41.977236 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.976833    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"
Nov 03 23:08:41.977236 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.976866    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" event={"ID":"0d8040b2-0353-449a-a8b6-01b6be112ad0","Type":"ContainerDied","Data":"6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2"}
Nov 03 23:08:41.977236 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.976918    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9" event={"ID":"0d8040b2-0353-449a-a8b6-01b6be112ad0","Type":"ContainerDied","Data":"9dbf7c7404527f1aa5abc8cfe8822f593d3a4561fb77123d1f4d6d189a9a227c"}
Nov 03 23:08:41.977236 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.976941    5647 scope.go:122] "RemoveContainer" containerID="6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2"
Nov 03 23:08:41.987401 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.987371    5647 scope.go:122] "RemoveContainer" containerID="6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5"
Nov 03 23:08:41.995575 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.995558    5647 scope.go:122] "RemoveContainer" containerID="6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2"
Nov 03 23:08:41.995858 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:41.995838    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2\": container with ID starting with 6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2 not found: ID does not exist" containerID="6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2"
Nov 03 23:08:41.995927 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.995868    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2"} err="failed to get container status \"6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2\": rpc error: code = NotFound desc = could not find container \"6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2\": container with ID starting with 6d875bfb5532b6000921bc175bf32199bfa32f050536d665ebe14aa19a7a47d2 not found: ID does not exist"
Nov 03 23:08:41.995927 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.995885    5647 scope.go:122] "RemoveContainer" containerID="6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5"
Nov 03 23:08:41.996162 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:41.996144    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5\": container with ID starting with 6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5 not found: ID does not exist" containerID="6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5"
Nov 03 23:08:41.996214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:41.996168    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5"} err="failed to get container status \"6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5\": rpc error: code = NotFound desc = could not find container \"6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5\": container with ID starting with 6bde8a7ecbd4b2650a710ad2edbc0ecac606d682ebe8700474300228370316b5 not found: ID does not exist"
Nov 03 23:08:42.149049 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:42.149015    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"]
Nov 03 23:08:42.195877 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:42.195844    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9"]
Nov 03 23:08:42.197458 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:42.197426    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"]
Nov 03 23:08:42.197707 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:42.197661    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy" containerID="cri-o://2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5" gracePeriod=30
Nov 03 23:08:42.694395 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:42.694361    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" path="/var/lib/kubelet/pods/0d8040b2-0353-449a-a8b6-01b6be112ad0/volumes"
Nov 03 23:08:46.041906 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:46.041862    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:08:47.454968 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.454930    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 23:08:47.542314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542224    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-certs\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542262    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-kube-api-access-8fb4m\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-kube-api-access-8fb4m\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542284    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d31b8fab-3824-47c6-95af-922b2359f74d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31b8fab-3824-47c6-95af-922b2359f74d-istiod-ca-cert\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542314 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542309    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-data\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542336    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-socket\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542361    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-envoy\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542436    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d31b8fab-3824-47c6-95af-922b2359f74d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31b8fab-3824-47c6-95af-922b2359f74d-istio-podinfo\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542468    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-istio-token\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-istio-token\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542507    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-credential-socket\") pod \"d31b8fab-3824-47c6-95af-922b2359f74d\" (UID: \"d31b8fab-3824-47c6-95af-922b2359f74d\") "
Nov 03 23:08:47.542642 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542624    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-certs" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:47.543053 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542716    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d31b8fab-3824-47c6-95af-922b2359f74d-istiod-ca-cert" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:08:47.543053 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542768    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-data" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:47.543053 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.542935    5647 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:47.543053 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.543021    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-socket" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:47.543053 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.543038    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-credential-socket" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:47.543261 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.543113    5647 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31b8fab-3824-47c6-95af-922b2359f74d-istiod-ca-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:47.543261 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.543131    5647 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-data\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:47.544828 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.544802    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-kube-api-access-8fb4m" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "kube-api-access-8fb4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:47.545127 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.545108    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-envoy" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 23:08:47.545227 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.545205    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-istio-token" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:08:47.545296 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.545269    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d31b8fab-3824-47c6-95af-922b2359f74d-istio-podinfo" pod "d31b8fab-3824-47c6-95af-922b2359f74d" (UID: "d31b8fab-3824-47c6-95af-922b2359f74d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 23:08:47.644131 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.644100    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fb4m\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-kube-api-access-8fb4m\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:47.644131 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.644130    5647 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-workload-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:47.644330 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.644142    5647 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-istio-envoy\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:47.644330 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.644150    5647 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31b8fab-3824-47c6-95af-922b2359f74d-istio-podinfo\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:47.644330 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.644158    5647 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d31b8fab-3824-47c6-95af-922b2359f74d-istio-token\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:47.644330 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:47.644166    5647 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31b8fab-3824-47c6-95af-922b2359f74d-credential-socket\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:08:48.012447 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.012410    5647 generic.go:356] "Generic (PLEG): container finished" podID="d31b8fab-3824-47c6-95af-922b2359f74d" containerID="2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5" exitCode=0
Nov 03 23:08:48.012627 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.012467    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"
Nov 03 23:08:48.012627 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.012497    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" event={"ID":"d31b8fab-3824-47c6-95af-922b2359f74d","Type":"ContainerDied","Data":"2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5"}
Nov 03 23:08:48.012627 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.012556    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r" event={"ID":"d31b8fab-3824-47c6-95af-922b2359f74d","Type":"ContainerDied","Data":"110f40b4ce6f43f2f4804f1edaf2e380d56e925046df5ef3f1b2716a2f9336b2"}
Nov 03 23:08:48.012627 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.012585    5647 scope.go:122] "RemoveContainer" containerID="2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5"
Nov 03 23:08:48.022172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.022155    5647 scope.go:122] "RemoveContainer" containerID="d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36"
Nov 03 23:08:48.030138 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.030120    5647 scope.go:122] "RemoveContainer" containerID="2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5"
Nov 03 23:08:48.030396 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:48.030377    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5\": container with ID starting with 2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5 not found: ID does not exist" containerID="2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5"
Nov 03 23:08:48.030458 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.030405    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5"} err="failed to get container status \"2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5\": rpc error: code = NotFound desc = could not find container \"2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5\": container with ID starting with 2b152166d54a6bf14077f49e639ce2e9cd7f20d2b74612980583bc069374bed5 not found: ID does not exist"
Nov 03 23:08:48.030458 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.030424    5647 scope.go:122] "RemoveContainer" containerID="d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36"
Nov 03 23:08:48.030657 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:08:48.030636    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36\": container with ID starting with d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36 not found: ID does not exist" containerID="d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36"
Nov 03 23:08:48.030792 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.030682    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36"} err="failed to get container status \"d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36\": rpc error: code = NotFound desc = could not find container \"d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36\": container with ID starting with d48fe237e50e4c680e4444ae36f7f73495597b86fc6d83cab58d8f6ccb0e4a36 not found: ID does not exist"
Nov 03 23:08:48.175369 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.175334    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"]
Nov 03 23:08:48.224840 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.224807    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r"]
Nov 03 23:08:48.694818 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.694784    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" path="/var/lib/kubelet/pods/d31b8fab-3824-47c6-95af-922b2359f74d/volumes"
Nov 03 23:08:48.824105 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:48.824071    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:08:49.112906 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.112861    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8657649675-nlgqt"]
Nov 03 23:08:49.113474 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113453    5647 cpu_manager.go:414] "containerMap: removing container" podUID="d0fb062c-7b7b-42c1-a99d-978d35c02be9" containerName="payload-processing"
Nov 03 23:08:49.113542 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113477    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0fb062c-7b7b-42c1-a99d-978d35c02be9" containerName="payload-processing"
Nov 03 23:08:49.113542 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113499    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a8e91d27-a1af-47df-8369-346bf5b8b7bc" containerName="revoke-keys"
Nov 03 23:08:49.113542 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113507    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8e91d27-a1af-47df-8369-346bf5b8b7bc" containerName="revoke-keys"
Nov 03 23:08:49.113542 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113517    5647 cpu_manager.go:414] "containerMap: removing container" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy"
Nov 03 23:08:49.113542 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113525    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy"
Nov 03 23:08:49.113542 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113542    5647 cpu_manager.go:414] "containerMap: removing container" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113549    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113559    5647 cpu_manager.go:414] "containerMap: removing container" podUID="dfd62750-e701-4dd2-8693-752677fafdd4" containerName="payload-pre-processing"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113566    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfd62750-e701-4dd2-8693-752677fafdd4" containerName="payload-pre-processing"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113578    5647 cpu_manager.go:414] "containerMap: removing container" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113585    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113621    5647 cpu_manager.go:414] "containerMap: removing container" podUID="70bb69e8-6434-42fd-b5cc-b1236f463311" containerName="revoke-keys"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113630    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70bb69e8-6434-42fd-b5cc-b1236f463311" containerName="revoke-keys"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113646    5647 cpu_manager.go:414] "containerMap: removing container" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113653    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113684    5647 cpu_manager.go:414] "containerMap: removing container" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113691    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113791    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0fb062c-7b7b-42c1-a99d-978d35c02be9" containerName="payload-processing"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113804    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8e91d27-a1af-47df-8369-346bf5b8b7bc" containerName="revoke-keys"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113814    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113836    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113844    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfd62750-e701-4dd2-8693-752677fafdd4" containerName="payload-pre-processing"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113859    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="70bb69e8-6434-42fd-b5cc-b1236f463311" containerName="revoke-keys"
Nov 03 23:08:49.113875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.113868    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="0315c1df-36d9-41a1-a3f9-581bb5aee4dd" containerName="main"
Nov 03 23:08:49.118979 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.118957    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.159352 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.159312    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-metrics-certs\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.159538 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.159388    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh882\" (UniqueName: \"kubernetes.io/projected/9588ad90-d811-4dab-a78d-2feb563caa98-kube-api-access-sh882\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.159538 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.159510    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-webhook-cert\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.220451 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.220418    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 03 23:08:49.220451 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.220421    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 03 23:08:49.220782 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.220418    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 03 23:08:49.220782 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.220490    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w82vk\""
Nov 03 23:08:49.222196 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.222174    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8657649675-nlgqt"]
Nov 03 23:08:49.260622 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.260589    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-metrics-certs\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.260832 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.260650    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh882\" (UniqueName: \"kubernetes.io/projected/9588ad90-d811-4dab-a78d-2feb563caa98-kube-api-access-sh882\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.260832 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.260726    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-webhook-cert\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.263310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.263278    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-metrics-certs\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.263310 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.263283    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-webhook-cert\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.321164 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.321133    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh882\" (UniqueName: \"kubernetes.io/projected/9588ad90-d811-4dab-a78d-2feb563caa98-kube-api-access-sh882\") pod \"maas-controller-8657649675-nlgqt\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") " pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.432656 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.432569    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:49.805427 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.805395    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8657649675-nlgqt"]
Nov 03 23:08:49.810168 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:49.810141    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 23:08:50.025055 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:50.025020    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8657649675-nlgqt" event={"ID":"9588ad90-d811-4dab-a78d-2feb563caa98","Type":"ContainerStarted","Data":"a3373bc17f5d55bd2ca940cbd65cc42d0de601e203e821fc1ff77ceffefa2ef5"}
Nov 03 23:08:52.037595 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:52.037558    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8657649675-nlgqt" event={"ID":"9588ad90-d811-4dab-a78d-2feb563caa98","Type":"ContainerStarted","Data":"1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1"}
Nov 03 23:08:52.038010 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:52.037620    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:08:52.195965 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:08:52.195914    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-8657649675-nlgqt" podStartSLOduration=1.036611115 podStartE2EDuration="3.195900691s" totalImagesPullingTime="2.159289576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:08:52.194571655 +0000 UTC m=+2755.635108913" watchObservedRunningTime="2026-11-03 23:08:52.195900691 +0000 UTC m=+2755.636437949"
Nov 03 23:09:03.045970 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:03.045939    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:09:04.816020 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.815987    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"]
Nov 03 23:09:04.816714 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.816660    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d8040b2-0353-449a-a8b6-01b6be112ad0" containerName="istio-proxy"
Nov 03 23:09:04.816830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.816730    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="d31b8fab-3824-47c6-95af-922b2359f74d" containerName="istio-proxy"
Nov 03 23:09:04.821423 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.821400    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.865866 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.865836    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"]
Nov 03 23:09:04.869334 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.869315    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"]
Nov 03 23:09:04.869439 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.869419    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:04.872904 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.872884    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:04.879696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.879643    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"]
Nov 03 23:09:04.883691 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.883653    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:04.905084 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905060    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.905204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905091    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.905204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905116    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.905204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905171    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.905313 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905299    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnv5v\" (UniqueName: \"kubernetes.io/projected/12c2c376-108d-4c3a-a7bd-b0420352ba4c-kube-api-access-tnv5v\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.905348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905324    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.905381 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905343    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.905416 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905396    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.905455 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.905438    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:04.913360 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.913334    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-89xv7\""
Nov 03 23:09:04.922120 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.922094    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"]
Nov 03 23:09:04.972503 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.972466    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"]
Nov 03 23:09:04.980487 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:04.980457    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"]
Nov 03 23:09:05.006482 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006446    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.006482 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006484    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.006723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006506    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.006723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006568    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.006723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006644    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.006723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006663    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.006723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006703    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.006889 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006753    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.006889 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006848    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.006889 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006879    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.006987 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006898    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007022 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.006994    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007057 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007026    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.007057 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007042    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ce51b1-83d6-4229-ae8c-07160de5deac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.007128 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007063    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.007128 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007093    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007200 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007125    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.007200 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007180    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.007282 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007251    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.007282 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007269    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007356 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007288    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxg4c\" (UniqueName: \"kubernetes.io/projected/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-kube-api-access-qxg4c\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.007356 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007309    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/171bf940-44d1-4211-9b51-4f4bc0207086-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.007432 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007362    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/171bf940-44d1-4211-9b51-4f4bc0207086-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.007432 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007376    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007432 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007395    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.007432 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007394    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007432 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007418    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnv5v\" (UniqueName: \"kubernetes.io/projected/12c2c376-108d-4c3a-a7bd-b0420352ba4c-kube-api-access-tnv5v\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007437    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.007605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007453    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm844\" (UniqueName: \"kubernetes.io/projected/171bf940-44d1-4211-9b51-4f4bc0207086-kube-api-access-nm844\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.007605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007471    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/171bf940-44d1-4211-9b51-4f4bc0207086-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.007605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007514    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007549    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.007605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007581    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007603    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007695    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.007867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007732    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.007867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007769    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g487z\" (UniqueName: \"kubernetes.io/projected/84ce51b1-83d6-4229-ae8c-07160de5deac-kube-api-access-g487z\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.007867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007803    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.007867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007803    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.008049 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.007906    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.008178 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.008161    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.010118 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.010089    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.010265 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.010247    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.041610 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.039030    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"]
Nov 03 23:09:05.068035 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.067966    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnv5v\" (UniqueName: \"kubernetes.io/projected/12c2c376-108d-4c3a-a7bd-b0420352ba4c-kube-api-access-tnv5v\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.068219 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.068139    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c2c376-108d-4c3a-a7bd-b0420352ba4c-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-qp7hb\" (UID: \"12c2c376-108d-4c3a-a7bd-b0420352ba4c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.108880 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.108844    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.108880 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.108883    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.109099 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.108920    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.109099 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.108952    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.109099 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.108984    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.109099 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109010    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109097    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109143    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109187    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109215    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109258    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109321    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109355    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ce51b1-83d6-4229-ae8c-07160de5deac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109377    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109382    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.109408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109396    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109402    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109454    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109461    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109465    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109484    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109516    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxg4c\" (UniqueName: \"kubernetes.io/projected/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-kube-api-access-qxg4c\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109546    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/171bf940-44d1-4211-9b51-4f4bc0207086-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109570    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/171bf940-44d1-4211-9b51-4f4bc0207086-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109594    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109638    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109699    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm844\" (UniqueName: \"kubernetes.io/projected/171bf940-44d1-4211-9b51-4f4bc0207086-kube-api-access-nm844\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109708    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109732    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/171bf940-44d1-4211-9b51-4f4bc0207086-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109761    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109766    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109819    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109866    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g487z\" (UniqueName: \"kubernetes.io/projected/84ce51b1-83d6-4229-ae8c-07160de5deac-kube-api-access-g487z\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.110002 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109878    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.111179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.110240    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.111179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.110597    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ce51b1-83d6-4229-ae8c-07160de5deac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.111179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.110758    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/171bf940-44d1-4211-9b51-4f4bc0207086-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.111179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.110776    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.111179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.109868    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-workload-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.111179 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.110836    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-data\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.111565 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.111540    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-workload-certs\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.111653 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.111581    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-credential-socket\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.112738 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.112709    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.112968 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.112932    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.113433 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.113414    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.113757 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.113738    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/171bf940-44d1-4211-9b51-4f4bc0207086-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.114074 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.114057    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.114163 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.114146    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/171bf940-44d1-4211-9b51-4f4bc0207086-istio-envoy\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.134319 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.134289    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:05.169863 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.169825    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.171087 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.171053    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g487z\" (UniqueName: \"kubernetes.io/projected/84ce51b1-83d6-4229-ae8c-07160de5deac-kube-api-access-g487z\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.171214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.171156    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxg4c\" (UniqueName: \"kubernetes.io/projected/9a0bcfb1-028b-4ac6-9432-7c10b040ff22-kube-api-access-qxg4c\") pod \"maas-default-gateway-openshift-default-7f5667f86c-l5859\" (UID: \"9a0bcfb1-028b-4ac6-9432-7c10b040ff22\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.171214 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.171160    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/171bf940-44d1-4211-9b51-4f4bc0207086-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.171529 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.171501    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ce51b1-83d6-4229-ae8c-07160de5deac-istio-token\") pod \"maas-default-gateway-openshift-default-7f5667f86c-fn2dg\" (UID: \"84ce51b1-83d6-4229-ae8c-07160de5deac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.171632 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.171617    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm844\" (UniqueName: \"kubernetes.io/projected/171bf940-44d1-4211-9b51-4f4bc0207086-kube-api-access-nm844\") pod \"maas-default-gateway-openshift-default-7f5667f86c-2z78m\" (UID: \"171bf940-44d1-4211-9b51-4f4bc0207086\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.185705 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.185402    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:05.192010 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.191980    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:05.198788 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.198226    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:05.560857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.560828    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"]
Nov 03 23:09:05.563096 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:09:05.563066    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c2c376_108d_4c3a_a7bd_b0420352ba4c.slice/crio-9edd33488a363669069aec981f6b9ad46bc10175684eb5b8a0bf2e3ae1eed1f8 WatchSource:0}: Error finding container 9edd33488a363669069aec981f6b9ad46bc10175684eb5b8a0bf2e3ae1eed1f8: Status 404 returned error can't find the container with id 9edd33488a363669069aec981f6b9ad46bc10175684eb5b8a0bf2e3ae1eed1f8
Nov 03 23:09:05.565187 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.565159    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.565271 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.565222    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.565271 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.565248    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.613596 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.613542    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"]
Nov 03 23:09:05.615772 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:09:05.615744    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171bf940_44d1_4211_9b51_4f4bc0207086.slice/crio-a00e48331230bfde7f31dc10dec25280e68e3e3d3bd2964fbc7ffde96df0654e WatchSource:0}: Error finding container a00e48331230bfde7f31dc10dec25280e68e3e3d3bd2964fbc7ffde96df0654e: Status 404 returned error can't find the container with id a00e48331230bfde7f31dc10dec25280e68e3e3d3bd2964fbc7ffde96df0654e
Nov 03 23:09:05.618059 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.618029    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.618152 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.618094    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.618152 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.618122    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.665739 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.665644    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"]
Nov 03 23:09:05.669067 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.668828    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"]
Nov 03 23:09:05.671859 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.671795    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.671993 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.671878    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.671993 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.671918    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.672482 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.672457    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.672579 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.672508    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:05.672579 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:05.672534    5647 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30891996Ki","pods":"250"}
Nov 03 23:09:06.134616 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.134577    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" event={"ID":"12c2c376-108d-4c3a-a7bd-b0420352ba4c","Type":"ContainerStarted","Data":"6bbc738be6e25354b2b78d4c7047679c32c8cbca6ade3a051543281229aac15c"}
Nov 03 23:09:06.135063 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.134632    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" event={"ID":"12c2c376-108d-4c3a-a7bd-b0420352ba4c","Type":"ContainerStarted","Data":"9edd33488a363669069aec981f6b9ad46bc10175684eb5b8a0bf2e3ae1eed1f8"}
Nov 03 23:09:06.135108 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.135067    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:06.136257 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.136233    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Nov 03 23:09:06.136384 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.136273    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Nov 03 23:09:06.137172 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.137145    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" event={"ID":"9a0bcfb1-028b-4ac6-9432-7c10b040ff22","Type":"ContainerStarted","Data":"d545e296a2d313472d35aa81cbe94ce96f321a3dc5272b383b297b7bf3e9edad"}
Nov 03 23:09:06.137213 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.137196    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" event={"ID":"9a0bcfb1-028b-4ac6-9432-7c10b040ff22","Type":"ContainerStarted","Data":"cc273d24be5dec22455c7b1a6fdc0bf2ce127db9999e481adebacb7cb9fb72bb"}
Nov 03 23:09:06.139662 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.139643    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" event={"ID":"171bf940-44d1-4211-9b51-4f4bc0207086","Type":"ContainerStarted","Data":"e0039012b01a2b5e0a2bf3eb9c58132e2b478b8e8ecf6fba2c4a7435b0caaa76"}
Nov 03 23:09:06.139745 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.139695    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" event={"ID":"171bf940-44d1-4211-9b51-4f4bc0207086","Type":"ContainerStarted","Data":"a00e48331230bfde7f31dc10dec25280e68e3e3d3bd2964fbc7ffde96df0654e"}
Nov 03 23:09:06.141978 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.141955    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" event={"ID":"84ce51b1-83d6-4229-ae8c-07160de5deac","Type":"ContainerStarted","Data":"9094c63328c56a2ff58c82849982f3dc36c8ea5d04e4d55d1c11a6426c48083c"}
Nov 03 23:09:06.142082 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.142037    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" event={"ID":"84ce51b1-83d6-4229-ae8c-07160de5deac","Type":"ContainerStarted","Data":"804761c74ffba8ebc73e57c89a2b2bcf234589c545d4874e6cfef96fafaae680"}
Nov 03 23:09:06.189055 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.189022    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:06.202696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.200996    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 03 23:09:06.202696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.201057    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 03 23:09:06.202696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.201126    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:06.202696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.201207    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Nov 03 23:09:06.202696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.201231    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Nov 03 23:09:06.208697 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.207080    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:06.231232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.231190    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Nov 03 23:09:06.231784 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.231577    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Nov 03 23:09:06.257826 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.257118    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podStartSLOduration=2.257097883 podStartE2EDuration="2.257097883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:09:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:09:06.2558172 +0000 UTC m=+2769.696354459" watchObservedRunningTime="2026-11-03 23:09:06.257097883 +0000 UTC m=+2769.697635142"
Nov 03 23:09:06.390758 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.377492    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podStartSLOduration=2.377472105 podStartE2EDuration="2.377472105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:09:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:09:06.376254037 +0000 UTC m=+2769.816791296" watchObservedRunningTime="2026-11-03 23:09:06.377472105 +0000 UTC m=+2769.818009365"
Nov 03 23:09:06.633786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.633721    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podStartSLOduration=2.6336982559999997 podStartE2EDuration="2.633698256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:09:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:09:06.507127759 +0000 UTC m=+2769.947665019" watchObservedRunningTime="2026-11-03 23:09:06.633698256 +0000 UTC m=+2770.074235513"
Nov 03 23:09:06.636033 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:06.635973    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podStartSLOduration=2.635955199 podStartE2EDuration="2.635955199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:09:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:09:06.632227454 +0000 UTC m=+2770.072764711" watchObservedRunningTime="2026-11-03 23:09:06.635955199 +0000 UTC m=+2770.076492459"
Nov 03 23:09:07.144112 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:07.144076    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Nov 03 23:09:07.149044 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:07.149002    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Nov 03 23:09:07.189197 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:07.189155    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 03 23:09:07.189505 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:07.189476    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 03 23:09:07.195082 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:07.195047    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Nov 03 23:09:07.195301 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:07.195276    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Nov 03 23:09:07.199051 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:07.199021    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Nov 03 23:09:07.199346 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:07.199319    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Nov 03 23:09:08.142804 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:08.142767    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Nov 03 23:09:08.143103 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:08.143073    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Nov 03 23:09:08.199694 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:08.199123    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Nov 03 23:09:08.199694 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:08.199187    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Nov 03 23:09:08.201195 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:08.201169    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Nov 03 23:09:08.201400 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:08.201376    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Nov 03 23:09:08.202486 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:08.202463    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 03 23:09:08.202683 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:08.202639    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 03 23:09:09.139504 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:09.139465    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Nov 03 23:09:09.139888 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:09.139857    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Nov 03 23:09:09.186276 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:09.186240    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 03 23:09:09.186621 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:09.186550    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 03 23:09:09.192802 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:09.192771    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Nov 03 23:09:09.192980 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:09.192955    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Nov 03 23:09:09.210080 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:09.210041    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Nov 03 23:09:09.210701 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:09.210639    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Nov 03 23:09:10.135440 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:10.135401    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Nov 03 23:09:10.135765 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:10.135729    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Nov 03 23:09:10.192187 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:10.192147    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 03 23:09:10.193554 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:10.193521    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 03 23:09:10.193820 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:10.193119    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Nov 03 23:09:10.193979 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:10.193959    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Nov 03 23:09:10.201620 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:10.201594    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Nov 03 23:09:10.201847 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:10.201822    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Nov 03 23:09:11.144701 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:11.139526    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Nov 03 23:09:11.144701 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:11.139588    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Nov 03 23:09:11.186697 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:11.186000    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 03 23:09:11.186697 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:11.186061    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 03 23:09:11.198694 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:11.195587    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Nov 03 23:09:11.198694 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:11.195647    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Nov 03 23:09:11.200039 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:11.200007    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Nov 03 23:09:11.200227 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:11.200198    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Nov 03 23:09:12.136474 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:12.136438    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Nov 03 23:09:12.136806 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:12.136773    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Nov 03 23:09:12.193809 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:12.193772    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Nov 03 23:09:12.194614 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:12.194586    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Nov 03 23:09:12.194806 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:12.194397    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 03 23:09:12.194921 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:12.194903    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 03 23:09:12.198957 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:12.198934    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Nov 03 23:09:12.199106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:12.199087    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Nov 03 23:09:13.135774 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:13.135728    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Nov 03 23:09:13.135978 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:13.135799    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Nov 03 23:09:13.186730 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:13.186640    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Nov 03 23:09:13.186958 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:13.186758    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Nov 03 23:09:13.192595 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:13.192559    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Nov 03 23:09:13.192810 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:13.192624    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Nov 03 23:09:13.199058 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:13.199017    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Nov 03 23:09:13.199581 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:13.199089    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Nov 03 23:09:15.190825 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.190784    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:09:15.191446 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.191410    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:09:15.232276 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.231847    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:09:15.232276 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.231915    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:09:15.253243 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.252784    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:09:15.253243 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.252855    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:09:15.260714 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.255610    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:09:15.260714 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.255659    5647 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:09:15.346597 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.344856    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:15.346597 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.346164    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:15.347501 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.347448    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m"
Nov 03 23:09:15.435364 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.435331    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:15.435705 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.435648    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:15.436539 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.436517    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg"
Nov 03 23:09:15.465819 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.465790    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:15.466164 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.466142    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:15.467096 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.467072    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb"
Nov 03 23:09:15.534786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.534752    5647 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:15.535144 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.535120    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:15.536092 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:15.536073    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859"
Nov 03 23:09:16.061405 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:09:16.061364    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:09:16.509132 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:16.509091    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:09:23.906010 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:23.905969    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8657649675-nlgqt"]
Nov 03 23:09:23.906594 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:23.906182    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-8657649675-nlgqt" podUID="9588ad90-d811-4dab-a78d-2feb563caa98" containerName="manager" containerID="cri-o://1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1" gracePeriod=30
Nov 03 23:09:24.159556 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.159494    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:09:24.248401 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.248365    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9588ad90-d811-4dab-a78d-2feb563caa98-kube-api-access-sh882\" (UniqueName: \"kubernetes.io/projected/9588ad90-d811-4dab-a78d-2feb563caa98-kube-api-access-sh882\") pod \"9588ad90-d811-4dab-a78d-2feb563caa98\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") "
Nov 03 23:09:24.248615 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.248423    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-webhook-cert\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-webhook-cert\") pod \"9588ad90-d811-4dab-a78d-2feb563caa98\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") "
Nov 03 23:09:24.248615 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.248501    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-metrics-certs\") pod \"9588ad90-d811-4dab-a78d-2feb563caa98\" (UID: \"9588ad90-d811-4dab-a78d-2feb563caa98\") "
Nov 03 23:09:24.250786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.250749    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-metrics-certs" pod "9588ad90-d811-4dab-a78d-2feb563caa98" (UID: "9588ad90-d811-4dab-a78d-2feb563caa98"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:09:24.250786 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.250771    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-webhook-cert" pod "9588ad90-d811-4dab-a78d-2feb563caa98" (UID: "9588ad90-d811-4dab-a78d-2feb563caa98"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:09:24.250935 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.250833    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9588ad90-d811-4dab-a78d-2feb563caa98-kube-api-access-sh882" pod "9588ad90-d811-4dab-a78d-2feb563caa98" (UID: "9588ad90-d811-4dab-a78d-2feb563caa98"). InnerVolumeSpecName "kube-api-access-sh882". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:09:24.349553 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.349518    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh882\" (UniqueName: \"kubernetes.io/projected/9588ad90-d811-4dab-a78d-2feb563caa98-kube-api-access-sh882\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:09:24.349553 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.349554    5647 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-webhook-cert\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:09:24.349822 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.349567    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9588ad90-d811-4dab-a78d-2feb563caa98-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:09:24.385928 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.385887    5647 generic.go:356] "Generic (PLEG): container finished" podID="9588ad90-d811-4dab-a78d-2feb563caa98" containerID="1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1" exitCode=0
Nov 03 23:09:24.386096 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.385975    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8657649675-nlgqt" event={"ID":"9588ad90-d811-4dab-a78d-2feb563caa98","Type":"ContainerDied","Data":"1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1"}
Nov 03 23:09:24.386096 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.386024    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8657649675-nlgqt" event={"ID":"9588ad90-d811-4dab-a78d-2feb563caa98","Type":"ContainerDied","Data":"a3373bc17f5d55bd2ca940cbd65cc42d0de601e203e821fc1ff77ceffefa2ef5"}
Nov 03 23:09:24.386096 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.386052    5647 scope.go:122] "RemoveContainer" containerID="1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1"
Nov 03 23:09:24.386231 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.386214    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8657649675-nlgqt"
Nov 03 23:09:24.397757 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.397729    5647 scope.go:122] "RemoveContainer" containerID="1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1"
Nov 03 23:09:24.398106 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:09:24.398085    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1\": container with ID starting with 1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1 not found: ID does not exist" containerID="1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1"
Nov 03 23:09:24.398199 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.398116    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1"} err="failed to get container status \"1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1\": rpc error: code = NotFound desc = could not find container \"1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1\": container with ID starting with 1c936ba65866df871721a63aea15e13e0011f32b0343f8d71878c937e150b5d1 not found: ID does not exist"
Nov 03 23:09:24.549656 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.549619    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8657649675-nlgqt"]
Nov 03 23:09:24.553898 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.553867    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8657649675-nlgqt"]
Nov 03 23:09:24.695232 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:24.695199    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9588ad90-d811-4dab-a78d-2feb563caa98" path="/var/lib/kubelet/pods/9588ad90-d811-4dab-a78d-2feb563caa98/volumes"
Nov 03 23:09:25.988699 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:25.988645    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8657649675-bqlwr"]
Nov 03 23:09:25.989377 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:25.989360    5647 cpu_manager.go:414] "containerMap: removing container" podUID="9588ad90-d811-4dab-a78d-2feb563caa98" containerName="manager"
Nov 03 23:09:25.989427 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:25.989380    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9588ad90-d811-4dab-a78d-2feb563caa98" containerName="manager"
Nov 03 23:09:25.989493 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:25.989482    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="9588ad90-d811-4dab-a78d-2feb563caa98" containerName="manager"
Nov 03 23:09:25.994460 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:25.994431    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.067136 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.067095    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-metrics-certs\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.067136 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.067142    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-webhook-cert\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.067424 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.067246    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll7rf\" (UniqueName: \"kubernetes.io/projected/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-kube-api-access-ll7rf\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.097000 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.096966    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 03 23:09:26.097329 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.096975    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w82vk\""
Nov 03 23:09:26.097421 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.097375    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 03 23:09:26.097461 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.097427    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 03 23:09:26.100094 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.100069    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8657649675-bqlwr"]
Nov 03 23:09:26.168868 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.168829    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-metrics-certs\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.168868 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.168872    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-webhook-cert\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.169131 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.168932    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll7rf\" (UniqueName: \"kubernetes.io/projected/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-kube-api-access-ll7rf\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.171544 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.171519    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-metrics-certs\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.171661 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.171540    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-webhook-cert\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.229276 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.229250    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll7rf\" (UniqueName: \"kubernetes.io/projected/5ad615c5-2739-4ec8-b0f5-c4ff640a2ade-kube-api-access-ll7rf\") pod \"maas-controller-8657649675-bqlwr\" (UID: \"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade\") " pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.306876 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.306786    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:26.680329 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:26.680290    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8657649675-bqlwr"]
Nov 03 23:09:26.682762 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:09:26.682730    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad615c5_2739_4ec8_b0f5_c4ff640a2ade.slice/crio-8b9d86b89d316178956c2a342ffda4d778c4dabccc1bb2e059edb471483be165 WatchSource:0}: Error finding container 8b9d86b89d316178956c2a342ffda4d778c4dabccc1bb2e059edb471483be165: Status 404 returned error can't find the container with id 8b9d86b89d316178956c2a342ffda4d778c4dabccc1bb2e059edb471483be165
Nov 03 23:09:27.404541 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:27.404507    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8657649675-bqlwr" event={"ID":"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade","Type":"ContainerStarted","Data":"8b9d86b89d316178956c2a342ffda4d778c4dabccc1bb2e059edb471483be165"}
Nov 03 23:09:28.412188 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:28.412150    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8657649675-bqlwr" event={"ID":"5ad615c5-2739-4ec8-b0f5-c4ff640a2ade","Type":"ContainerStarted","Data":"7bdc29b1d6c0b296eb15b362c75a40d4f065f22d0c04caacc4b6265bb4b6d91b"}
Nov 03 23:09:28.412646 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:28.412286    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:28.532606 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:28.532543    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-8657649675-bqlwr" podStartSLOduration=2.682401396 podStartE2EDuration="3.53252822s" totalImagesPullingTime="850.126824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:09:28.531075048 +0000 UTC m=+2791.971612308" watchObservedRunningTime="2026-11-03 23:09:28.53252822 +0000 UTC m=+2791.973065478"
Nov 03 23:09:39.420221 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:09:39.420182    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8657649675-bqlwr"
Nov 03 23:09:46.043222 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:09:46.043178    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:10:11.192783 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:10:11.192746    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:10:16.041901 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:10:16.041867    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:10:32.801122 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:10:32.801088    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:10:43.579451 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:10:43.579411    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:10:46.042166 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:10:46.042127    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:11:04.083852 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:11:04.083819    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:11:16.043588 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:11:16.043483    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:11:20.962648 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:11:20.962610    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:11:46.043046 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:11:46.043002    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:11:52.171737 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:11:52.171702    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:12:02.784218 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:12:02.784184    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:12:15.186311 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:12:15.186268    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:12:16.041858 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:12:16.041820    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:12:37.506148 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:12:37.506065    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:12:46.041464 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:12:46.041415    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:12:47.062297 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:12:47.062259    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:12:57.143005 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:12:57.142967    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:12:57.152883 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:12:57.152854    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:13:05.511639 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:05.511599    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:13:12.046963 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:12.046925    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"]
Nov 03 23:13:12.096331 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:12.096296    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f5b27b8f-c6xhj"]
Nov 03 23:13:12.699451 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:12.699408    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1538d524-6411-47f9-a8b6-fa97d897873a" path="/var/lib/kubelet/pods/1538d524-6411-47f9-a8b6-fa97d897873a/volumes"
Nov 03 23:13:16.042768 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:13:16.042725    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:13:18.053616 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:18.053575    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"]
Nov 03 23:13:18.103713 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:18.103648    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"]
Nov 03 23:13:18.106022 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:18.105996    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61b6d326-q6q2m"]
Nov 03 23:13:18.107385 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:18.107352    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4288120c-cvn6p"]
Nov 03 23:13:18.108558 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:18.108535    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:13:18.695451 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:18.695415    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70bb69e8-6434-42fd-b5cc-b1236f463311" path="/var/lib/kubelet/pods/70bb69e8-6434-42fd-b5cc-b1236f463311/volumes"
Nov 03 23:13:18.695868 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:18.695854    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8e91d27-a1af-47df-8369-346bf5b8b7bc" path="/var/lib/kubelet/pods/a8e91d27-a1af-47df-8369-346bf5b8b7bc/volumes"
Nov 03 23:13:37.906513 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:37.906474    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:13:46.043091 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:13:46.043054    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:13:59.984755 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:13:59.984722    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:14:01.489849 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:01.489803    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:14:01.490541 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:01.490490    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:14:01.514841 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:01.514802    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:14:01.515161 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:01.515115    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:14:01.586878 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:01.586837    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:14:01.587259 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:01.587226    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:14:12.984710 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:12.984587    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:14:16.083994 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:14:16.083936    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:14:23.470640 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:23.470600    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:14:43.505089 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:43.505042    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:14:46.047374 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:14:46.047332    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:14:53.468964 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:53.468923    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:14:59.991749 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:14:59.991714    5647 scope.go:122] "RemoveContainer" containerID="a2e3829b392b929aa9580dc8cfe423b44c037e872ca14650fccc27285b52d2f2"
Nov 03 23:15:00.001007 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:15:00.000978    5647 scope.go:122] "RemoveContainer" containerID="69bb53e29bdcc8b6128b0c3a55ee8333d2530aae5105c9d742ac81178d78eac8"
Nov 03 23:15:00.010019 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:15:00.009995    5647 scope.go:122] "RemoveContainer" containerID="2f66d5f05f9b5317217ca8d44a56b53e4325a46e0d621df7920c38231192e416"
Nov 03 23:15:09.185906 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:15:09.185869    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:15:16.041763 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:15:16.041719    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:15:32.184076 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:15:32.184038    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:15:42.906140 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:15:42.906100    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:15:46.041917 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:15:46.041880    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:16:01.300764 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:01.300729    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:16:11.684355 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:11.684321    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:16:16.042945 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:16:16.042907    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:16:29.368247 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.368215    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"]
Nov 03 23:16:29.371849 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.371832    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.425169 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.425143    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-8a134f-serving-cert\""
Nov 03 23:16:29.474957 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.474924    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"]
Nov 03 23:16:29.482064 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.482035    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-metrics-certs\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.482194 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.482090    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gjqd\" (UniqueName: \"kubernetes.io/projected/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-kube-api-access-4gjqd\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.482194 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.482157    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.482277 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.482191    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.582797 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.582764    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-metrics-certs\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.582985 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.582824    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gjqd\" (UniqueName: \"kubernetes.io/projected/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-kube-api-access-4gjqd\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.582985 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.582841    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.582985 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.582861    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.583159 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:16:29.582982    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-serving-cert: secret "maas-api-e2e-worker-single-main-8a134f-serving-cert" not found
Nov 03 23:16:29.583159 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:16:29.583057    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls podName:03a2264a-f6a0-4fa8-9a86-4fd7666919a1 nodeName:}" failed. No retries permitted until 2026-11-03 23:16:30.083036427 +0000 UTC m=+3213.523573681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls") pod "maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj" (UID: "03a2264a-f6a0-4fa8-9a86-4fd7666919a1") : secret "maas-api-e2e-worker-single-main-8a134f-serving-cert" not found
Nov 03 23:16:29.583491 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.583472    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.585449 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.585427    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-metrics-certs\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:29.644036 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:29.643965    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gjqd\" (UniqueName: \"kubernetes.io/projected/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-kube-api-access-4gjqd\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:30.093004 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:30.092962    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:30.095551 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:30.095530    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:30.285140 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:30.285101    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:30.658422 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:30.658396    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"]
Nov 03 23:16:30.661328 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:16:30.661299    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a2264a_f6a0_4fa8_9a86_4fd7666919a1.slice/crio-457cc45f5987593d83e59fcd24f972a5fbc3689cec978214d8ca9bd3c0ebc5ed WatchSource:0}: Error finding container 457cc45f5987593d83e59fcd24f972a5fbc3689cec978214d8ca9bd3c0ebc5ed: Status 404 returned error can't find the container with id 457cc45f5987593d83e59fcd24f972a5fbc3689cec978214d8ca9bd3c0ebc5ed
Nov 03 23:16:30.662827 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:30.662807    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 23:16:31.006311 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:31.006276    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj" event={"ID":"03a2264a-f6a0-4fa8-9a86-4fd7666919a1","Type":"ContainerStarted","Data":"457cc45f5987593d83e59fcd24f972a5fbc3689cec978214d8ca9bd3c0ebc5ed"}
Nov 03 23:16:32.013175 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:32.013141    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj" event={"ID":"03a2264a-f6a0-4fa8-9a86-4fd7666919a1","Type":"ContainerStarted","Data":"27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709"}
Nov 03 23:16:32.013579 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:32.013239    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:32.130916 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:32.130865    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj" podStartSLOduration=2.217561759 podStartE2EDuration="3.13085176s" totalImagesPullingTime="913.290001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:16:32.129945654 +0000 UTC m=+3215.570482914" watchObservedRunningTime="2026-11-03 23:16:32.13085176 +0000 UTC m=+3215.571389019"
Nov 03 23:16:37.409062 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:37.409025    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:16:38.024319 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:38.024290    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:16:46.042758 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:16:46.042717    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:16:56.295158 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:16:56.295126    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:17:16.042900 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:17:16.042811    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:17:24.199857 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:17:24.199814    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:17:46.043659 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:17:46.043621    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:17:53.364749 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:17:53.364712    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:17:57.182026 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:17:57.181993    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:17:57.192016 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:17:57.191992    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:18:05.882372 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:05.882337    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"]
Nov 03 23:18:05.882843 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:05.882535    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj" podUID="03a2264a-f6a0-4fa8-9a86-4fd7666919a1" containerName="maas-api" containerID="cri-o://27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709" gracePeriod=30
Nov 03 23:18:07.129278 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.129251    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:18:07.247348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.247310    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-kube-api-access-4gjqd\" (UniqueName: \"kubernetes.io/projected/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-kube-api-access-4gjqd\") pod \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") "
Nov 03 23:18:07.247553 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.247393    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-openshift-service-ca\") pod \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") "
Nov 03 23:18:07.247553 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.247513    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-metrics-certs\") pod \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") "
Nov 03 23:18:07.247644 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.247608    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls\") pod \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\" (UID: \"03a2264a-f6a0-4fa8-9a86-4fd7666919a1\") "
Nov 03 23:18:07.248195 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.248150    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-openshift-service-ca" pod "03a2264a-f6a0-4fa8-9a86-4fd7666919a1" (UID: "03a2264a-f6a0-4fa8-9a86-4fd7666919a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:18:07.250739 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.250531    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-metrics-certs" pod "03a2264a-f6a0-4fa8-9a86-4fd7666919a1" (UID: "03a2264a-f6a0-4fa8-9a86-4fd7666919a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:18:07.250739 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.250644    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-kube-api-access-4gjqd" pod "03a2264a-f6a0-4fa8-9a86-4fd7666919a1" (UID: "03a2264a-f6a0-4fa8-9a86-4fd7666919a1"). InnerVolumeSpecName "kube-api-access-4gjqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:18:07.251307 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.251281    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls" pod "03a2264a-f6a0-4fa8-9a86-4fd7666919a1" (UID: "03a2264a-f6a0-4fa8-9a86-4fd7666919a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:18:07.349644 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.349609    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:18:07.349644 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.349638    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:18:07.349644 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.349648    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:18:07.349899 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.349656    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gjqd\" (UniqueName: \"kubernetes.io/projected/03a2264a-f6a0-4fa8-9a86-4fd7666919a1-kube-api-access-4gjqd\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:18:07.502430 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.502344    5647 generic.go:356] "Generic (PLEG): container finished" podID="03a2264a-f6a0-4fa8-9a86-4fd7666919a1" containerID="27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709" exitCode=0
Nov 03 23:18:07.502592 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.502428    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj" event={"ID":"03a2264a-f6a0-4fa8-9a86-4fd7666919a1","Type":"ContainerDied","Data":"27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709"}
Nov 03 23:18:07.502592 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.502494    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj" event={"ID":"03a2264a-f6a0-4fa8-9a86-4fd7666919a1","Type":"ContainerDied","Data":"457cc45f5987593d83e59fcd24f972a5fbc3689cec978214d8ca9bd3c0ebc5ed"}
Nov 03 23:18:07.502592 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.502519    5647 scope.go:122] "RemoveContainer" containerID="27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709"
Nov 03 23:18:07.502592 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.502446    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"
Nov 03 23:18:07.512236 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.512217    5647 scope.go:122] "RemoveContainer" containerID="27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709"
Nov 03 23:18:07.512488 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:18:07.512467    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709\": container with ID starting with 27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709 not found: ID does not exist" containerID="27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709"
Nov 03 23:18:07.512557 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.512495    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709"} err="failed to get container status \"27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709\": rpc error: code = NotFound desc = could not find container \"27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709\": container with ID starting with 27cce69bf8abb29839a082d5b903c818ac50302583d7bd11a2941f3128446709 not found: ID does not exist"
Nov 03 23:18:07.665240 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.665208    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"]
Nov 03 23:18:07.671303 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:07.671265    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-8a134f-584d99986c-wh7qj"]
Nov 03 23:18:08.695177 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:08.695140    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03a2264a-f6a0-4fa8-9a86-4fd7666919a1" path="/var/lib/kubelet/pods/03a2264a-f6a0-4fa8-9a86-4fd7666919a1/volumes"
Nov 03 23:18:10.683970 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:10.683691    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:18:16.046709 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:18:16.046650    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:18:31.161816 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:31.161781    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:18:41.702074 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:18:41.702040    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:18:46.042697 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:18:46.042637    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:19:16.043524 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:19:16.043484    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:19:46.042201 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:19:46.042166    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:20:16.042855 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:20:16.042761    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:20:18.194597 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:20:18.194555    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:20:35.466780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:20:35.466740    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:20:44.894830 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:20:44.894783    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:20:46.042651 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:20:46.042615    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:21:04.416779 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:04.416739    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:21:15.245040 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.244998    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"]
Nov 03 23:21:15.245690 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.245647    5647 cpu_manager.go:414] "containerMap: removing container" podUID="03a2264a-f6a0-4fa8-9a86-4fd7666919a1" containerName="maas-api"
Nov 03 23:21:15.245801 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.245689    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03a2264a-f6a0-4fa8-9a86-4fd7666919a1" containerName="maas-api"
Nov 03 23:21:15.245860 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.245821    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="03a2264a-f6a0-4fa8-9a86-4fd7666919a1" containerName="maas-api"
Nov 03 23:21:15.249057 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.249037    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.301131 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.301088    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-60ef36-serving-cert\""
Nov 03 23:21:15.311325 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.311294    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"]
Nov 03 23:21:15.365031 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.364999    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-metrics-certs\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.365220 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.365059    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.365220 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.365153    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h7jh\" (UniqueName: \"kubernetes.io/projected/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-kube-api-access-6h7jh\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.365336 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.365245    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.466510 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.466464    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.466738 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.466521    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h7jh\" (UniqueName: \"kubernetes.io/projected/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-kube-api-access-6h7jh\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.466738 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.466585    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.466738 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:21:15.466598    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-serving-cert: secret "maas-api-e2e-worker-single-main-60ef36-serving-cert" not found
Nov 03 23:21:15.466738 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:21:15.466663    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls podName:b7c3ce5d-9a87-40d6-9ac5-23564299ad79 nodeName:}" failed. No retries permitted until 2026-11-03 23:21:15.966647466 +0000 UTC m=+3499.407184703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls") pod "maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl" (UID: "b7c3ce5d-9a87-40d6-9ac5-23564299ad79") : secret "maas-api-e2e-worker-single-main-60ef36-serving-cert" not found
Nov 03 23:21:15.467071 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.466751    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-metrics-certs\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.467307 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.467288    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.469406 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.469386    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-metrics-certs\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.528008 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.527925    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h7jh\" (UniqueName: \"kubernetes.io/projected/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-kube-api-access-6h7jh\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.973758 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.973718    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:15.976412 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:15.976382    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:16.041159 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:21:16.041122    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:21:16.161681 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:16.161647    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:16.537937 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:16.537914    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"]
Nov 03 23:21:16.540479 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:21:16.540453    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c3ce5d_9a87_40d6_9ac5_23564299ad79.slice/crio-c1ac202c34ed09e938fbd666be056606b090dd8dc7b19b5d1a537d7b22fbd0f6 WatchSource:0}: Error finding container c1ac202c34ed09e938fbd666be056606b090dd8dc7b19b5d1a537d7b22fbd0f6: Status 404 returned error can't find the container with id c1ac202c34ed09e938fbd666be056606b090dd8dc7b19b5d1a537d7b22fbd0f6
Nov 03 23:21:17.526564 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:17.526526    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl" event={"ID":"b7c3ce5d-9a87-40d6-9ac5-23564299ad79","Type":"ContainerStarted","Data":"45f58dffbfc03735e112afab5dc863a07151c22167a5a636bae0b4e8b6eafc68"}
Nov 03 23:21:17.526806 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:17.526581    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl" event={"ID":"b7c3ce5d-9a87-40d6-9ac5-23564299ad79","Type":"ContainerStarted","Data":"c1ac202c34ed09e938fbd666be056606b090dd8dc7b19b5d1a537d7b22fbd0f6"}
Nov 03 23:21:17.526806 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:17.526711    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:17.654836 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:17.654734    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl" podStartSLOduration=1.821810511 podStartE2EDuration="2.654720078s" totalImagesPullingTime="832.909567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:21:17.652651679 +0000 UTC m=+3501.093188965" watchObservedRunningTime="2026-11-03 23:21:17.654720078 +0000 UTC m=+3501.095257337"
Nov 03 23:21:23.394348 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:23.394310    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:21:23.537633 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:23.537605    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:21:42.482486 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:21:42.482447    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:21:46.043407 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:21:46.043367    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:22:09.995469 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:09.995429    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:22:16.042787 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:22:16.042747    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:22:35.703507 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:35.703472    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:22:46.050141 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:22:46.050096    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:22:51.789759 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:51.789718    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"]
Nov 03 23:22:51.790284 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:51.789961    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl" podUID="b7c3ce5d-9a87-40d6-9ac5-23564299ad79" containerName="maas-api" containerID="cri-o://45f58dffbfc03735e112afab5dc863a07151c22167a5a636bae0b4e8b6eafc68" gracePeriod=30
Nov 03 23:22:53.001839 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.001788    5647 generic.go:356] "Generic (PLEG): container finished" podID="b7c3ce5d-9a87-40d6-9ac5-23564299ad79" containerID="45f58dffbfc03735e112afab5dc863a07151c22167a5a636bae0b4e8b6eafc68" exitCode=0
Nov 03 23:22:53.002313 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.001870    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl" event={"ID":"b7c3ce5d-9a87-40d6-9ac5-23564299ad79","Type":"ContainerDied","Data":"45f58dffbfc03735e112afab5dc863a07151c22167a5a636bae0b4e8b6eafc68"}
Nov 03 23:22:53.057620 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.057595    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:22:53.162419 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.162325    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-metrics-certs\") pod \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") "
Nov 03 23:22:53.162419 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.162389    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-kube-api-access-6h7jh\" (UniqueName: \"kubernetes.io/projected/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-kube-api-access-6h7jh\") pod \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") "
Nov 03 23:22:53.162712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.162558    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-openshift-service-ca\") pod \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") "
Nov 03 23:22:53.162712 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.162611    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls\") pod \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\" (UID: \"b7c3ce5d-9a87-40d6-9ac5-23564299ad79\") "
Nov 03 23:22:53.162944 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.162925    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-openshift-service-ca" pod "b7c3ce5d-9a87-40d6-9ac5-23564299ad79" (UID: "b7c3ce5d-9a87-40d6-9ac5-23564299ad79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:22:53.163118 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.163080    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:22:53.164794 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.164764    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-kube-api-access-6h7jh" pod "b7c3ce5d-9a87-40d6-9ac5-23564299ad79" (UID: "b7c3ce5d-9a87-40d6-9ac5-23564299ad79"). InnerVolumeSpecName "kube-api-access-6h7jh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:22:53.164794 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.164765    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-metrics-certs" pod "b7c3ce5d-9a87-40d6-9ac5-23564299ad79" (UID: "b7c3ce5d-9a87-40d6-9ac5-23564299ad79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:22:53.164957 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.164864    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls" pod "b7c3ce5d-9a87-40d6-9ac5-23564299ad79" (UID: "b7c3ce5d-9a87-40d6-9ac5-23564299ad79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:22:53.264686 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.264642    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:22:53.264879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.264708    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h7jh\" (UniqueName: \"kubernetes.io/projected/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-kube-api-access-6h7jh\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:22:53.264879 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:53.264719    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7c3ce5d-9a87-40d6-9ac5-23564299ad79-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:22:54.008364 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:54.008331    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl" event={"ID":"b7c3ce5d-9a87-40d6-9ac5-23564299ad79","Type":"ContainerDied","Data":"c1ac202c34ed09e938fbd666be056606b090dd8dc7b19b5d1a537d7b22fbd0f6"}
Nov 03 23:22:54.008840 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:54.008371    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"
Nov 03 23:22:54.008840 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:54.008396    5647 scope.go:122] "RemoveContainer" containerID="45f58dffbfc03735e112afab5dc863a07151c22167a5a636bae0b4e8b6eafc68"
Nov 03 23:22:54.173371 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:54.173340    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"]
Nov 03 23:22:54.222618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:54.222584    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60ef36-5cf797f7d-4qbjl"]
Nov 03 23:22:54.694859 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:54.694824    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7c3ce5d-9a87-40d6-9ac5-23564299ad79" path="/var/lib/kubelet/pods/b7c3ce5d-9a87-40d6-9ac5-23564299ad79/volumes"
Nov 03 23:22:56.470580 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:56.470548    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:22:57.214609 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:57.214584    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:22:57.227338 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:22:57.227319    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:23:11.864305 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:23:11.864209    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:23:16.042107 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:23:16.042068    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:23:46.042651 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:23:46.042613    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:23:53.607819 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:23:53.607784    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:24:16.041874 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:24:16.041832    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:24:25.700139 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:24:25.700099    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:24:40.974806 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:24:40.974772    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:24:46.042430 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:24:46.042392    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:24:53.203522 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:24:53.203485    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:25:01.402831 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:01.402788    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:25:01.403451 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:01.403414    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:25:01.508918 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:01.508877    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:25:01.509235 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:01.509202    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg" podUID="84ce51b1-83d6-4229-ae8c-07160de5deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:25:01.513802 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:01.513765    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:25:01.514012 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:01.513984    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb" podUID="12c2c376-108d-4c3a-a7bd-b0420352ba4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:25:01.583696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:01.582792    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:25:01.583696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:01.582865    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859" podUID="9a0bcfb1-028b-4ac6-9432-7c10b040ff22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:25:03.483008 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:03.482972    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:25:13.104060 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:13.104024    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:25:16.133924 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:25:16.133781    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:25:36.386783 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:36.386748    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:25:36.867930 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:36.867897    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"]
Nov 03 23:25:36.868393 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:36.868378    5647 cpu_manager.go:414] "containerMap: removing container" podUID="b7c3ce5d-9a87-40d6-9ac5-23564299ad79" containerName="maas-api"
Nov 03 23:25:36.868466 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:36.868394    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7c3ce5d-9a87-40d6-9ac5-23564299ad79" containerName="maas-api"
Nov 03 23:25:36.868509 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:36.868466    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7c3ce5d-9a87-40d6-9ac5-23564299ad79" containerName="maas-api"
Nov 03 23:25:36.871709 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:36.871690    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:36.974868 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:36.974838    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"]
Nov 03 23:25:37.015983 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.015953    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b0549b1-d737-415e-8dba-2d3f2df0af38-metrics-certs\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.016204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.016021    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqdvd\" (UniqueName: \"kubernetes.io/projected/3b0549b1-d737-415e-8dba-2d3f2df0af38-kube-api-access-qqdvd\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.016204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.016135    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b0549b1-d737-415e-8dba-2d3f2df0af38-maas-api-tls\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.016204 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.016191    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b0549b1-d737-415e-8dba-2d3f2df0af38-openshift-service-ca\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.116726 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.116693    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b0549b1-d737-415e-8dba-2d3f2df0af38-metrics-certs\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.116939 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.116759    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqdvd\" (UniqueName: \"kubernetes.io/projected/3b0549b1-d737-415e-8dba-2d3f2df0af38-kube-api-access-qqdvd\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.116939 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.116806    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b0549b1-d737-415e-8dba-2d3f2df0af38-maas-api-tls\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.116939 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.116822    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b0549b1-d737-415e-8dba-2d3f2df0af38-openshift-service-ca\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.117587 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.117555    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b0549b1-d737-415e-8dba-2d3f2df0af38-openshift-service-ca\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.119404 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.119353    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b0549b1-d737-415e-8dba-2d3f2df0af38-maas-api-tls\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.119532 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.119509    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b0549b1-d737-415e-8dba-2d3f2df0af38-metrics-certs\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.177474 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.177439    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqdvd\" (UniqueName: \"kubernetes.io/projected/3b0549b1-d737-415e-8dba-2d3f2df0af38-kube-api-access-qqdvd\") pod \"maas-api-6c76b5d8d7-498zz\" (UID: \"3b0549b1-d737-415e-8dba-2d3f2df0af38\") " pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.185303 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.185283    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:37.557307 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.557275    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"]
Nov 03 23:25:37.559964 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:25:37.559936    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b0549b1_d737_415e_8dba_2d3f2df0af38.slice/crio-d80e575b1ab7759461edbba7c63cc39d85b6c1ffa15311596b37b8640c9a6bee WatchSource:0}: Error finding container d80e575b1ab7759461edbba7c63cc39d85b6c1ffa15311596b37b8640c9a6bee: Status 404 returned error can't find the container with id d80e575b1ab7759461edbba7c63cc39d85b6c1ffa15311596b37b8640c9a6bee
Nov 03 23:25:37.561279 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:37.561258    5647 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 23:25:38.023304 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:38.023267    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz" event={"ID":"3b0549b1-d737-415e-8dba-2d3f2df0af38","Type":"ContainerStarted","Data":"d80e575b1ab7759461edbba7c63cc39d85b6c1ffa15311596b37b8640c9a6bee"}
Nov 03 23:25:39.030486 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:39.030446    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz" event={"ID":"3b0549b1-d737-415e-8dba-2d3f2df0af38","Type":"ContainerStarted","Data":"2af991bace000a689c08447170fd2762a203b97319168cc1bda1e1af5001cc1d"}
Nov 03 23:25:39.030952 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:39.030561    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:39.147686 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:39.147609    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz" podStartSLOduration=2.205338644 podStartE2EDuration="3.147593493s" totalImagesPullingTime="942.254849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:25:39.145969567 +0000 UTC m=+3762.586506827" watchObservedRunningTime="2026-11-03 23:25:39.147593493 +0000 UTC m=+3762.588130765"
Nov 03 23:25:45.041577 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:45.041546    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6c76b5d8d7-498zz"
Nov 03 23:25:46.043261 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:25:46.043217    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:25:52.289106 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.289071    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"]
Nov 03 23:25:52.292918 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.292900    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.385158 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.385110    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-metrics-certs\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.385358 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.385173    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.385358 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.385260    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-openshift-service-ca\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.385358 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.385283    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rx2t\" (UniqueName: \"kubernetes.io/projected/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-kube-api-access-5rx2t\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.389248 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.389221    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-738d0c9a-serving-cert\""
Nov 03 23:25:52.390942 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.390921    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"]
Nov 03 23:25:52.486241 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.486209    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-metrics-certs\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.486420 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.486264    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.486420 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.486305    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-openshift-service-ca\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.486420 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.486330    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rx2t\" (UniqueName: \"kubernetes.io/projected/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-kube-api-access-5rx2t\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.486420 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:25:52.486374    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-serving-cert: secret "maas-api-e2e-ait-738d0c9a-serving-cert" not found
Nov 03 23:25:52.486587 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:25:52.486440    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls podName:8dbd0f39-a137-42c3-8e29-0abbe7c031ef nodeName:}" failed. No retries permitted until 2026-11-03 23:25:52.98641947 +0000 UTC m=+3776.426956709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls") pod "maas-api-e2e-ait-738d0c9a-796866ffd8-plndp" (UID: "8dbd0f39-a137-42c3-8e29-0abbe7c031ef") : secret "maas-api-e2e-ait-738d0c9a-serving-cert" not found
Nov 03 23:25:52.487275 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.487251    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-openshift-service-ca\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.488861 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.488838    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-metrics-certs\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.549403 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.549333    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rx2t\" (UniqueName: \"kubernetes.io/projected/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-kube-api-access-5rx2t\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.993253 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.993216    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:52.995775 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:52.995756    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls\") pod \"maas-api-e2e-ait-738d0c9a-796866ffd8-plndp\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:53.206296 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:53.206258    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:53.445378 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:53.445345    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"]
Nov 03 23:25:53.445824 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:53.445550    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg" podUID="a14e804a-62c7-468b-b465-ec62bff41bbb" containerName="maas-api" containerID="cri-o://a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112" gracePeriod=30
Nov 03 23:25:53.603172 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:25:53.603136    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbd0f39_a137_42c3_8e29_0abbe7c031ef.slice/crio-8e39aa43d6253dc9357ee4855121565a9ea8c4c050702bb2ad40dabde92c28b5 WatchSource:0}: Error finding container 8e39aa43d6253dc9357ee4855121565a9ea8c4c050702bb2ad40dabde92c28b5: Status 404 returned error can't find the container with id 8e39aa43d6253dc9357ee4855121565a9ea8c4c050702bb2ad40dabde92c28b5
Nov 03 23:25:53.612852 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:53.612827    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"]
Nov 03 23:25:54.124095 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.124060    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp" event={"ID":"8dbd0f39-a137-42c3-8e29-0abbe7c031ef","Type":"ContainerStarted","Data":"8e39aa43d6253dc9357ee4855121565a9ea8c4c050702bb2ad40dabde92c28b5"}
Nov 03 23:25:54.436478 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.436442    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg" podUID="a14e804a-62c7-468b-b465-ec62bff41bbb" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.30:8443/health\": dial tcp 10.134.0.30:8443: connect: connection refused"
Nov 03 23:25:54.716900 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.716876    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 23:25:54.809954 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.809922    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-maas-api-tls\") pod \"a14e804a-62c7-468b-b465-ec62bff41bbb\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") "
Nov 03 23:25:54.810129 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.810025    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a14e804a-62c7-468b-b465-ec62bff41bbb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14e804a-62c7-468b-b465-ec62bff41bbb-openshift-service-ca\") pod \"a14e804a-62c7-468b-b465-ec62bff41bbb\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") "
Nov 03 23:25:54.810129 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.810074    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-metrics-certs\") pod \"a14e804a-62c7-468b-b465-ec62bff41bbb\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") "
Nov 03 23:25:54.810129 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.810112    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a14e804a-62c7-468b-b465-ec62bff41bbb-kube-api-access-kmctt\" (UniqueName: \"kubernetes.io/projected/a14e804a-62c7-468b-b465-ec62bff41bbb-kube-api-access-kmctt\") pod \"a14e804a-62c7-468b-b465-ec62bff41bbb\" (UID: \"a14e804a-62c7-468b-b465-ec62bff41bbb\") "
Nov 03 23:25:54.810441 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.810411    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a14e804a-62c7-468b-b465-ec62bff41bbb-openshift-service-ca" pod "a14e804a-62c7-468b-b465-ec62bff41bbb" (UID: "a14e804a-62c7-468b-b465-ec62bff41bbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:25:54.812361 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.812340    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-metrics-certs" pod "a14e804a-62c7-468b-b465-ec62bff41bbb" (UID: "a14e804a-62c7-468b-b465-ec62bff41bbb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:25:54.812945 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.812928    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a14e804a-62c7-468b-b465-ec62bff41bbb-kube-api-access-kmctt" pod "a14e804a-62c7-468b-b465-ec62bff41bbb" (UID: "a14e804a-62c7-468b-b465-ec62bff41bbb"). InnerVolumeSpecName "kube-api-access-kmctt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:25:54.813010 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.812935    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-maas-api-tls" pod "a14e804a-62c7-468b-b465-ec62bff41bbb" (UID: "a14e804a-62c7-468b-b465-ec62bff41bbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:25:54.911441 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.911408    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14e804a-62c7-468b-b465-ec62bff41bbb-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:25:54.911441 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.911435    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:25:54.911441 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.911447    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmctt\" (UniqueName: \"kubernetes.io/projected/a14e804a-62c7-468b-b465-ec62bff41bbb-kube-api-access-kmctt\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:25:54.911706 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:54.911456    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14e804a-62c7-468b-b465-ec62bff41bbb-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:25:55.130276 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.130240    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp" event={"ID":"8dbd0f39-a137-42c3-8e29-0abbe7c031ef","Type":"ContainerStarted","Data":"8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c"}
Nov 03 23:25:55.130506 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.130304    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:25:55.131540 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.131516    5647 generic.go:356] "Generic (PLEG): container finished" podID="a14e804a-62c7-468b-b465-ec62bff41bbb" containerID="a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112" exitCode=0
Nov 03 23:25:55.131643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.131550    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"
Nov 03 23:25:55.131643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.131555    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg" event={"ID":"a14e804a-62c7-468b-b465-ec62bff41bbb","Type":"ContainerDied","Data":"a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112"}
Nov 03 23:25:55.131643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.131593    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg" event={"ID":"a14e804a-62c7-468b-b465-ec62bff41bbb","Type":"ContainerDied","Data":"98e9f3eb9ca5529d00eb2c32b53bef2dd7ea98a38b50c0a72d0cd2ad54305af5"}
Nov 03 23:25:55.131643 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.131612    5647 scope.go:122] "RemoveContainer" containerID="a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112"
Nov 03 23:25:55.141700 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.141659    5647 scope.go:122] "RemoveContainer" containerID="a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112"
Nov 03 23:25:55.141944 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:25:55.141925    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112\": container with ID starting with a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112 not found: ID does not exist" containerID="a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112"
Nov 03 23:25:55.142009 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.141952    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112"} err="failed to get container status \"a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112\": rpc error: code = NotFound desc = could not find container \"a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112\": container with ID starting with a6dff464f2f47630d3f70eaf39662e605d8f5e4b6c0c5ef7e1dd763f9ea53112 not found: ID does not exist"
Nov 03 23:25:55.248372 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.248319    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp" podStartSLOduration=2.448286407 podStartE2EDuration="3.248304569s" totalImagesPullingTime="800.018162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:25:55.245848505 +0000 UTC m=+3778.686385763" watchObservedRunningTime="2026-11-03 23:25:55.248304569 +0000 UTC m=+3778.688841828"
Nov 03 23:25:55.365130 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.362420    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"]
Nov 03 23:25:55.365130 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:55.362479    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddf589cff-94hxg"]
Nov 03 23:25:56.695686 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:25:56.695639    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a14e804a-62c7-468b-b465-ec62bff41bbb" path="/var/lib/kubelet/pods/a14e804a-62c7-468b-b465-ec62bff41bbb/volumes"
Nov 03 23:26:00.664081 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:00.664028    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:26:01.147427 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:01.147396    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:26:16.041956 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:26:16.041872    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:26:31.499236 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.499197    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"]
Nov 03 23:26:31.500163 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.499786    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp" podUID="8dbd0f39-a137-42c3-8e29-0abbe7c031ef" containerName="maas-api" containerID="cri-o://8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c" gracePeriod=30
Nov 03 23:26:31.782160 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.782129    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:26:31.919890 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.919252    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-kube-api-access-5rx2t\" (UniqueName: \"kubernetes.io/projected/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-kube-api-access-5rx2t\") pod \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") "
Nov 03 23:26:31.919890 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.919327    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-metrics-certs\") pod \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") "
Nov 03 23:26:31.919890 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.919462    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls\") pod \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") "
Nov 03 23:26:31.919890 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.919532    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-openshift-service-ca\") pod \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\" (UID: \"8dbd0f39-a137-42c3-8e29-0abbe7c031ef\") "
Nov 03 23:26:31.920473 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.920450    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-openshift-service-ca" pod "8dbd0f39-a137-42c3-8e29-0abbe7c031ef" (UID: "8dbd0f39-a137-42c3-8e29-0abbe7c031ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:26:31.924723 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.923661    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls" pod "8dbd0f39-a137-42c3-8e29-0abbe7c031ef" (UID: "8dbd0f39-a137-42c3-8e29-0abbe7c031ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:26:31.930256 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.930219    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-metrics-certs" pod "8dbd0f39-a137-42c3-8e29-0abbe7c031ef" (UID: "8dbd0f39-a137-42c3-8e29-0abbe7c031ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:26:31.931154 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:31.931124    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-kube-api-access-5rx2t" pod "8dbd0f39-a137-42c3-8e29-0abbe7c031ef" (UID: "8dbd0f39-a137-42c3-8e29-0abbe7c031ef"). InnerVolumeSpecName "kube-api-access-5rx2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:26:32.025239 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.025080    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:26:32.025239 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.025123    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:26:32.025239 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.025139    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rx2t\" (UniqueName: \"kubernetes.io/projected/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-kube-api-access-5rx2t\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:26:32.025239 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.025154    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbd0f39-a137-42c3-8e29-0abbe7c031ef-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:26:32.318612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.318527    5647 generic.go:356] "Generic (PLEG): container finished" podID="8dbd0f39-a137-42c3-8e29-0abbe7c031ef" containerID="8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c" exitCode=0
Nov 03 23:26:32.318612 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.318591    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"
Nov 03 23:26:32.318838 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.318618    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp" event={"ID":"8dbd0f39-a137-42c3-8e29-0abbe7c031ef","Type":"ContainerDied","Data":"8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c"}
Nov 03 23:26:32.318838 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.318730    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp" event={"ID":"8dbd0f39-a137-42c3-8e29-0abbe7c031ef","Type":"ContainerDied","Data":"8e39aa43d6253dc9357ee4855121565a9ea8c4c050702bb2ad40dabde92c28b5"}
Nov 03 23:26:32.318838 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.318755    5647 scope.go:122] "RemoveContainer" containerID="8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c"
Nov 03 23:26:32.328713 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.328688    5647 scope.go:122] "RemoveContainer" containerID="8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c"
Nov 03 23:26:32.328990 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:26:32.328972    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c\": container with ID starting with 8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c not found: ID does not exist" containerID="8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c"
Nov 03 23:26:32.329063 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.329004    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c"} err="failed to get container status \"8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c\": rpc error: code = NotFound desc = could not find container \"8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c\": container with ID starting with 8f96866dfa0966709cb8b262a102622965042684d17ded86a2fd10c50f02de0c not found: ID does not exist"
Nov 03 23:26:32.490265 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.490224    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"]
Nov 03 23:26:32.493990 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.493965    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-738d0c9a-796866ffd8-plndp"]
Nov 03 23:26:32.695495 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:32.695415    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8dbd0f39-a137-42c3-8e29-0abbe7c031ef" path="/var/lib/kubelet/pods/8dbd0f39-a137-42c3-8e29-0abbe7c031ef/volumes"
Nov 03 23:26:38.077696 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:38.077629    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:26:43.386618 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.386575    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"]
Nov 03 23:26:43.387108 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.387091    5647 cpu_manager.go:414] "containerMap: removing container" podUID="a14e804a-62c7-468b-b465-ec62bff41bbb" containerName="maas-api"
Nov 03 23:26:43.387167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.387109    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a14e804a-62c7-468b-b465-ec62bff41bbb" containerName="maas-api"
Nov 03 23:26:43.387167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.387157    5647 cpu_manager.go:414] "containerMap: removing container" podUID="8dbd0f39-a137-42c3-8e29-0abbe7c031ef" containerName="maas-api"
Nov 03 23:26:43.387167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.387162    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8dbd0f39-a137-42c3-8e29-0abbe7c031ef" containerName="maas-api"
Nov 03 23:26:43.387279 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.387222    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="8dbd0f39-a137-42c3-8e29-0abbe7c031ef" containerName="maas-api"
Nov 03 23:26:43.387279 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.387230    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="a14e804a-62c7-468b-b465-ec62bff41bbb" containerName="maas-api"
Nov 03 23:26:43.391886 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.391865    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.446811 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.446771    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-034b52-serving-cert\""
Nov 03 23:26:43.497550 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.497510    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"]
Nov 03 23:26:43.552297 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.552255    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxdjk\" (UniqueName: \"kubernetes.io/projected/986f99d8-efc7-4a0a-9935-66a496e6ca46-kube-api-access-xxdjk\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.552508 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.552423    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/986f99d8-efc7-4a0a-9935-66a496e6ca46-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.552564 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.552517    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.552564 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.552548    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-metrics-certs\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.654171 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.654083    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/986f99d8-efc7-4a0a-9935-66a496e6ca46-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.654171 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.654153    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.654379 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.654193    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-metrics-certs\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.654379 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.654270    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxdjk\" (UniqueName: \"kubernetes.io/projected/986f99d8-efc7-4a0a-9935-66a496e6ca46-kube-api-access-xxdjk\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.654379 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:26:43.654311    5647 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-serving-cert: secret "maas-api-e2e-authcfg-http-034b52-serving-cert" not found
Nov 03 23:26:43.654538 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:26:43.654390    5647 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls podName:986f99d8-efc7-4a0a-9935-66a496e6ca46 nodeName:}" failed. No retries permitted until 2026-11-03 23:26:44.154367072 +0000 UTC m=+3827.594904312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls") pod "maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl" (UID: "986f99d8-efc7-4a0a-9935-66a496e6ca46") : secret "maas-api-e2e-authcfg-http-034b52-serving-cert" not found
Nov 03 23:26:43.654807 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.654789    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/986f99d8-efc7-4a0a-9935-66a496e6ca46-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.656904 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.656886    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-metrics-certs\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:43.715035 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:43.714999    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxdjk\" (UniqueName: \"kubernetes.io/projected/986f99d8-efc7-4a0a-9935-66a496e6ca46-kube-api-access-xxdjk\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:44.158581 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:44.158545    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:44.161207 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:44.161180    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:44.304893 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:44.304855    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:44.681863 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:44.681773    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"]
Nov 03 23:26:44.684269 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:26:44.684243    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986f99d8_efc7_4a0a_9935_66a496e6ca46.slice/crio-96a78290ffa4bd010ff5592e77f3a9b85ade485496b15671131fbe47a5d71d09 WatchSource:0}: Error finding container 96a78290ffa4bd010ff5592e77f3a9b85ade485496b15671131fbe47a5d71d09: Status 404 returned error can't find the container with id 96a78290ffa4bd010ff5592e77f3a9b85ade485496b15671131fbe47a5d71d09
Nov 03 23:26:45.385072 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:45.385039    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl" event={"ID":"986f99d8-efc7-4a0a-9935-66a496e6ca46","Type":"ContainerStarted","Data":"96a78290ffa4bd010ff5592e77f3a9b85ade485496b15671131fbe47a5d71d09"}
Nov 03 23:26:46.043639 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:26:46.043599    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:26:46.391598 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:46.391502    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl" event={"ID":"986f99d8-efc7-4a0a-9935-66a496e6ca46","Type":"ContainerStarted","Data":"2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74"}
Nov 03 23:26:46.391861 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:46.391692    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:26:46.511321 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:46.511263    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl" podStartSLOduration=2.515097823 podStartE2EDuration="3.511247541s" totalImagesPullingTime="996.149718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 23:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:26:46.509545193 +0000 UTC m=+3829.950082465" watchObservedRunningTime="2026-11-03 23:26:46.511247541 +0000 UTC m=+3829.951784799"
Nov 03 23:26:50.076630 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:50.076593    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:26:52.401733 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:26:52.401701    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:27:16.043303 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:27:16.043267    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:27:46.042741 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:27:46.042696    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:27:57.248361 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:27:57.248329    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:27:57.261958 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:27:57.261936    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:28:16.042593 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:28:16.042559    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:28:46.042895 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:28:46.042857    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:28:53.946799 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:53.946761    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"]
Nov 03 23:28:53.947356 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:53.946957    5647 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl" podUID="986f99d8-efc7-4a0a-9935-66a496e6ca46" containerName="maas-api" containerID="cri-o://2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74" gracePeriod=30
Nov 03 23:28:54.239516 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.239490    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:28:54.387969 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.387936    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/986f99d8-efc7-4a0a-9935-66a496e6ca46-kube-api-access-xxdjk\" (UniqueName: \"kubernetes.io/projected/986f99d8-efc7-4a0a-9935-66a496e6ca46-kube-api-access-xxdjk\") pod \"986f99d8-efc7-4a0a-9935-66a496e6ca46\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") "
Nov 03 23:28:54.388143 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.387980    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-metrics-certs\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-metrics-certs\") pod \"986f99d8-efc7-4a0a-9935-66a496e6ca46\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") "
Nov 03 23:28:54.388143 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.388133    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/986f99d8-efc7-4a0a-9935-66a496e6ca46-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/986f99d8-efc7-4a0a-9935-66a496e6ca46-openshift-service-ca\") pod \"986f99d8-efc7-4a0a-9935-66a496e6ca46\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") "
Nov 03 23:28:54.388254 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.388183    5647 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls\") pod \"986f99d8-efc7-4a0a-9935-66a496e6ca46\" (UID: \"986f99d8-efc7-4a0a-9935-66a496e6ca46\") "
Nov 03 23:28:54.388571 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.388539    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/986f99d8-efc7-4a0a-9935-66a496e6ca46-openshift-service-ca" pod "986f99d8-efc7-4a0a-9935-66a496e6ca46" (UID: "986f99d8-efc7-4a0a-9935-66a496e6ca46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 23:28:54.390378 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.390357    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls" pod "986f99d8-efc7-4a0a-9935-66a496e6ca46" (UID: "986f99d8-efc7-4a0a-9935-66a496e6ca46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:28:54.390622 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.390607    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/986f99d8-efc7-4a0a-9935-66a496e6ca46-kube-api-access-xxdjk" pod "986f99d8-efc7-4a0a-9935-66a496e6ca46" (UID: "986f99d8-efc7-4a0a-9935-66a496e6ca46"). InnerVolumeSpecName "kube-api-access-xxdjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 23:28:54.391768 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.391746    5647 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-metrics-certs" pod "986f99d8-efc7-4a0a-9935-66a496e6ca46" (UID: "986f99d8-efc7-4a0a-9935-66a496e6ca46"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 23:28:54.489436 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.489314    5647 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-maas-api-tls\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:28:54.489436 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.489371    5647 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxdjk\" (UniqueName: \"kubernetes.io/projected/986f99d8-efc7-4a0a-9935-66a496e6ca46-kube-api-access-xxdjk\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:28:54.489436 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.489433    5647 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/986f99d8-efc7-4a0a-9935-66a496e6ca46-metrics-certs\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:28:54.489710 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:54.489456    5647 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/986f99d8-efc7-4a0a-9935-66a496e6ca46-openshift-service-ca\") on node \"ip-10-0-132-187.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 23:28:55.031299 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.031261    5647 generic.go:356] "Generic (PLEG): container finished" podID="986f99d8-efc7-4a0a-9935-66a496e6ca46" containerID="2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74" exitCode=0
Nov 03 23:28:55.031859 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.031309    5647 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"
Nov 03 23:28:55.031859 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.031336    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl" event={"ID":"986f99d8-efc7-4a0a-9935-66a496e6ca46","Type":"ContainerDied","Data":"2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74"}
Nov 03 23:28:55.031859 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.031380    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl" event={"ID":"986f99d8-efc7-4a0a-9935-66a496e6ca46","Type":"ContainerDied","Data":"96a78290ffa4bd010ff5592e77f3a9b85ade485496b15671131fbe47a5d71d09"}
Nov 03 23:28:55.031859 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.031403    5647 scope.go:122] "RemoveContainer" containerID="2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74"
Nov 03 23:28:55.040823 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.040794    5647 scope.go:122] "RemoveContainer" containerID="2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74"
Nov 03 23:28:55.041115 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:28:55.041093    5647 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74\": container with ID starting with 2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74 not found: ID does not exist" containerID="2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74"
Nov 03 23:28:55.041233 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.041130    5647 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74"} err="failed to get container status \"2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74\": rpc error: code = NotFound desc = could not find container \"2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74\": container with ID starting with 2d683ada57c7281f81750de9a67ee310155a4ccbe2de8558d441a79bd0bf3a74 not found: ID does not exist"
Nov 03 23:28:55.194120 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.194091    5647 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"]
Nov 03 23:28:55.243329 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:55.243296    5647 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-034b52-5b6cfb677b-hltzl"]
Nov 03 23:28:56.695784 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:56.695753    5647 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="986f99d8-efc7-4a0a-9935-66a496e6ca46" path="/var/lib/kubelet/pods/986f99d8-efc7-4a0a-9935-66a496e6ca46/volumes"
Nov 03 23:28:59.200489 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:28:59.200450    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:29:09.390997 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:29:09.390903    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:29:16.042835 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:29:16.042794    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:29:22.287430 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:29:22.287388    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:29:43.297049 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:29:43.297011    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:29:46.042739 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:29:46.042702    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:30:14.107534 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:14.107492    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-4lqhk"]
Nov 03 23:30:16.156949 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:30:16.156906    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:30:16.393528 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:16.393490    5647 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 23:30:16.393895 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:16.393864    5647 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m" podUID="171bf940-44d1-4211-9b51-4f4bc0207086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 23:30:24.078844 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:24.078815    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5d6776c6bc-p8lvw_b455b70d-d28f-48ac-a4ef-7a5c7392d06f/manager/1.log"
Nov 03 23:30:25.933521 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:25.933486    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8657649675-bqlwr_5ad615c5-2739-4ec8-b0f5-c4ff640a2ade/manager/0.log"
Nov 03 23:30:26.295891 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:26.295852    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-66wzs_bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da/server/0.log"
Nov 03 23:30:26.508989 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:26.508941    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-xmkpx_60dcf158-4b80-418b-98bb-53e5f3b0a179/manager/0.log"
Nov 03 23:30:27.229839 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:27.229811    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5d6776c6bc-p8lvw_b455b70d-d28f-48ac-a4ef-7a5c7392d06f/manager/0.log"
Nov 03 23:30:27.802418 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:27.802389    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6c76b5d8d7-498zz_3b0549b1-d737-415e-8dba-2d3f2df0af38/maas-api/0.log"
Nov 03 23:30:32.282326 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:32.282293    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-4lqhk_891f78e8-3535-4eea-9912-b93f9fc571c3/limitador/0.log"
Nov 03 23:30:33.630602 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:33.630575    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-cwdgf_2ff0e28c-db6e-4176-b124-a79a6f78ba51/discovery/0.log"
Nov 03 23:30:33.935014 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:33.934928    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-586b8ddf6d-h9m6s_82ca6b39-a72a-4632-9342-10eca3d67829/kube-auth-proxy/0.log"
Nov 03 23:30:34.367802 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:34.367771    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-2z78m_171bf940-44d1-4211-9b51-4f4bc0207086/istio-proxy/0.log"
Nov 03 23:30:35.342303 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:35.342269    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-fn2dg_84ce51b1-83d6-4229-ae8c-07160de5deac/istio-proxy/0.log"
Nov 03 23:30:35.608263 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:35.608184    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-l5859_9a0bcfb1-028b-4ac6-9432-7c10b040ff22/istio-proxy/0.log"
Nov 03 23:30:36.112160 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:36.112126    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-qp7hb_12c2c376-108d-4c3a-a7bd-b0420352ba4c/istio-proxy/0.log"
Nov 03 23:30:37.077741 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:37.077650    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-vxr7w_cf4f293e-6230-416c-ac10-2820e7c01188/payload-pre-processing/0.log"
Nov 03 23:30:37.308502 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:37.308475    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wqbz4_ba296c09-65ef-4276-be45-96373f18b09a/payload-processing/0.log"
Nov 03 23:30:38.418145 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:38.418114    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj_1b27706b-6628-4cc7-812e-e22541de8064/main/0.log"
Nov 03 23:30:38.910077 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:38.910051    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-c7zcs_f01a335e-fe1d-40cd-87ca-8611d04c3404/main/0.log"
Nov 03 23:30:39.351038 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:39.351010    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q_90a88463-27e7-4a04-86d7-ef9ba10d9a08/main/0.log"
Nov 03 23:30:39.843593 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:39.843562    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75_4fd7ac8b-8570-444e-ad0b-66ed03b79fa2/main/0.log"
Nov 03 23:30:46.042411 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:30:46.042371    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:30:50.160858 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:50.160829    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5d6776c6bc-p8lvw_b455b70d-d28f-48ac-a4ef-7a5c7392d06f/manager/1.log"
Nov 03 23:30:51.610259 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:51.610231    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8657649675-bqlwr_5ad615c5-2739-4ec8-b0f5-c4ff640a2ade/manager/0.log"
Nov 03 23:30:52.013619 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:52.013586    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-66wzs_bdf9054c-8dea-49d6-a7e2-c9c6fd02c3da/server/0.log"
Nov 03 23:30:52.235889 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:52.235856    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-xmkpx_60dcf158-4b80-418b-98bb-53e5f3b0a179/manager/0.log"
Nov 03 23:30:53.101988 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:53.101961    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6c76b5d8d7-498zz_3b0549b1-d737-415e-8dba-2d3f2df0af38/maas-api/0.log"
Nov 03 23:30:57.170003 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:57.169975    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-4lqhk_891f78e8-3535-4eea-9912-b93f9fc571c3/limitador/0.log"
Nov 03 23:30:58.233614 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:58.233579    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-cwdgf_2ff0e28c-db6e-4176-b124-a79a6f78ba51/discovery/0.log"
Nov 03 23:30:58.548279 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:58.548203    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-586b8ddf6d-h9m6s_82ca6b39-a72a-4632-9342-10eca3d67829/kube-auth-proxy/0.log"
Nov 03 23:30:59.013626 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:30:59.013600    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-2z78m_171bf940-44d1-4211-9b51-4f4bc0207086/istio-proxy/0.log"
Nov 03 23:31:00.061063 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:00.061031    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-fn2dg_84ce51b1-83d6-4229-ae8c-07160de5deac/istio-proxy/0.log"
Nov 03 23:31:00.332605 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:00.332521    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-l5859_9a0bcfb1-028b-4ac6-9432-7c10b040ff22/istio-proxy/0.log"
Nov 03 23:31:00.820347 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:00.820314    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-qp7hb_12c2c376-108d-4c3a-a7bd-b0420352ba4c/istio-proxy/0.log"
Nov 03 23:31:01.773471 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:01.773441    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-vxr7w_cf4f293e-6230-416c-ac10-2820e7c01188/payload-pre-processing/0.log"
Nov 03 23:31:01.977151 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:01.977122    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wqbz4_ba296c09-65ef-4276-be45-96373f18b09a/payload-processing/0.log"
Nov 03 23:31:02.843284 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:02.843255    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-mcfmj_1b27706b-6628-4cc7-812e-e22541de8064/main/0.log"
Nov 03 23:31:03.387368 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:03.387335    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-c7zcs_f01a335e-fe1d-40cd-87ca-8611d04c3404/main/0.log"
Nov 03 23:31:03.887301 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:03.887272    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dr28q_90a88463-27e7-4a04-86d7-ef9ba10d9a08/main/0.log"
Nov 03 23:31:04.378822 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:04.378791    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-dfb75_4fd7ac8b-8570-444e-ad0b-66ed03b79fa2/main/0.log"
Nov 03 23:31:16.041040 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:31:16.040998    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:31:33.656194 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:33.656156    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8657649675-bqlwr_5ad615c5-2739-4ec8-b0f5-c4ff640a2ade/manager/0.log"
Nov 03 23:31:34.152377 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:34.152341    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8657649675-bqlwr_5ad615c5-2739-4ec8-b0f5-c4ff640a2ade/manager/0.log"
Nov 03 23:31:34.907025 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:34.906998    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wqbz4_ba296c09-65ef-4276-be45-96373f18b09a/payload-processing/0.log"
Nov 03 23:31:35.134144 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:31:35.134116    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wqbz4_ba296c09-65ef-4276-be45-96373f18b09a/payload-processing/0.log"
Nov 03 23:31:46.043642 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:31:46.043597    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:32:12.504217 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:12.504133    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8657649675-bqlwr_5ad615c5-2739-4ec8-b0f5-c4ff640a2ade/manager/0.log"
Nov 03 23:32:13.078639 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:13.078595    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8657649675-bqlwr_5ad615c5-2739-4ec8-b0f5-c4ff640a2ade/manager/0.log"
Nov 03 23:32:13.819221 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:13.819188    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wqbz4_ba296c09-65ef-4276-be45-96373f18b09a/payload-processing/0.log"
Nov 03 23:32:14.024047 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:14.024020    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wqbz4_ba296c09-65ef-4276-be45-96373f18b09a/payload-processing/0.log"
Nov 03 23:32:16.044555 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:32:16.044517    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:32:24.408780 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:24.408735    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zhsp8_fd5c8298-1f37-4c24-bb52-1536b0e93028/global-pull-secret-syncer/0.log"
Nov 03 23:32:27.386059 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:27.386021    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qj6mk_0b0deac8-c535-4d85-b27e-968b22b55c14/konnectivity-agent/0.log"
Nov 03 23:32:27.924525 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:27.924492    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-187.us-west-1.compute.internal_9cc67bfe6e75d748c4abbd6f6df49a40/haproxy/0.log"
Nov 03 23:32:33.830613 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.830572    5647 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"]
Nov 03 23:32:33.831055 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.831043    5647 cpu_manager.go:414] "containerMap: removing container" podUID="986f99d8-efc7-4a0a-9935-66a496e6ca46" containerName="maas-api"
Nov 03 23:32:33.831100 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.831056    5647 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="986f99d8-efc7-4a0a-9935-66a496e6ca46" containerName="maas-api"
Nov 03 23:32:33.831167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.831157    5647 memory_manager.go:365] "RemoveStaleState removing state" podUID="986f99d8-efc7-4a0a-9935-66a496e6ca46" containerName="maas-api"
Nov 03 23:32:33.834374 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.834357    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:33.886449 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.886416    5647 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x9d6v\"/\"default-dockercfg-z6rsm\""
Nov 03 23:32:33.922580 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.922543    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht428\" (UniqueName: \"kubernetes.io/projected/3619aa47-862e-4658-80ee-635fbc25c1ef-kube-api-access-ht428\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:33.922580 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.922591    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-podres\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:33.922992 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.922793    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-proc\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:33.922992 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.922850    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-lib-modules\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:33.922992 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.922877    5647 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-sys\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:33.938820 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.938782    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"]
Nov 03 23:32:33.981875 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.981833    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x9d6v\"/\"openshift-service-ca.crt\""
Nov 03 23:32:33.982064 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:33.981932    5647 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x9d6v\"/\"kube-root-ca.crt\""
Nov 03 23:32:34.023576 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023536    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-lib-modules\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.023576 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023581    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-sys\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.023867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023621    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht428\" (UniqueName: \"kubernetes.io/projected/3619aa47-862e-4658-80ee-635fbc25c1ef-kube-api-access-ht428\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.023867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023638    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-podres\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.023867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023723    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-lib-modules\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.023867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023729    5647 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-proc\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.023867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023763    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-sys\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.023867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023776    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-proc\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.023867 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.023802    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3619aa47-862e-4658-80ee-635fbc25c1ef-podres\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.084469 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.084382    5647 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht428\" (UniqueName: \"kubernetes.io/projected/3619aa47-862e-4658-80ee-635fbc25c1ef-kube-api-access-ht428\") pod \"perf-node-gather-daemonset-jfrfn\" (UID: \"3619aa47-862e-4658-80ee-635fbc25c1ef\") " pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.147127 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.147090    5647 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:34.521991 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:34.521958    5647 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"]
Nov 03 23:32:34.524793 ip-10-0-132-187 kubenswrapper[5647]: W1103 23:32:34.524764    5647 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3619aa47_862e_4658_80ee_635fbc25c1ef.slice/crio-67a5f9fc150d440b817553133fcc80ac907e6be07655f0a11692e0e986bf4f97 WatchSource:0}: Error finding container 67a5f9fc150d440b817553133fcc80ac907e6be07655f0a11692e0e986bf4f97: Status 404 returned error can't find the container with id 67a5f9fc150d440b817553133fcc80ac907e6be07655f0a11692e0e986bf4f97
Nov 03 23:32:35.294760 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:35.294726    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn" event={"ID":"3619aa47-862e-4658-80ee-635fbc25c1ef","Type":"ContainerStarted","Data":"8c645fa59a2203bfdf727047af64f17b04fc5c82e345220d88beed99136dad51"}
Nov 03 23:32:35.295191 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:35.294786    5647 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn" event={"ID":"3619aa47-862e-4658-80ee-635fbc25c1ef","Type":"ContainerStarted","Data":"67a5f9fc150d440b817553133fcc80ac907e6be07655f0a11692e0e986bf4f97"}
Nov 03 23:32:35.295191 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:35.294824    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:35.410971 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:35.410911    5647 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn" podStartSLOduration=2.410895947 podStartE2EDuration="2.410895947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 23:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 23:32:35.409904212 +0000 UTC m=+4178.850441473" watchObservedRunningTime="2026-11-03 23:32:35.410895947 +0000 UTC m=+4178.851433205"
Nov 03 23:32:35.773460 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:35.773429    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-4lqhk_891f78e8-3535-4eea-9912-b93f9fc571c3/limitador/0.log"
Nov 03 23:32:38.793167 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:38.793136    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_36f9a8e0-049b-40e4-b79b-cf5b9bf28149/alertmanager/0.log"
Nov 03 23:32:39.206933 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:39.206906    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_36f9a8e0-049b-40e4-b79b-cf5b9bf28149/config-reloader/0.log"
Nov 03 23:32:39.620702 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:39.620604    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_36f9a8e0-049b-40e4-b79b-cf5b9bf28149/kube-rbac-proxy-web/0.log"
Nov 03 23:32:40.037317 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:40.037289    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_36f9a8e0-049b-40e4-b79b-cf5b9bf28149/kube-rbac-proxy/0.log"
Nov 03 23:32:40.450888 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:40.450809    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_36f9a8e0-049b-40e4-b79b-cf5b9bf28149/kube-rbac-proxy-metric/0.log"
Nov 03 23:32:40.864566 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:40.864539    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_36f9a8e0-049b-40e4-b79b-cf5b9bf28149/prom-label-proxy/0.log"
Nov 03 23:32:41.279808 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:41.279776    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_36f9a8e0-049b-40e4-b79b-cf5b9bf28149/init-config-reloader/0.log"
Nov 03 23:32:41.311408 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:41.311376    5647 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x9d6v/perf-node-gather-daemonset-jfrfn"
Nov 03 23:32:42.170271 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:42.170238    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-tslkb_9e91ab02-2291-4937-b729-0422ee58256b/kube-state-metrics/0.log"
Nov 03 23:32:42.579010 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:42.578977    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-tslkb_9e91ab02-2291-4937-b729-0422ee58256b/kube-rbac-proxy-main/0.log"
Nov 03 23:32:42.994294 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:42.994271    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-tslkb_9e91ab02-2291-4937-b729-0422ee58256b/kube-rbac-proxy-self/0.log"
Nov 03 23:32:43.414903 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:43.414828    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6ddc555bd7-sljjr_4196c457-6d8d-4a01-b299-653d0b4e4445/metrics-server/0.log"
Nov 03 23:32:44.254759 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:44.254724    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8rxz5_5db69683-41ca-4f88-a7ab-5a1174170725/node-exporter/0.log"
Nov 03 23:32:44.668776 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:44.668697    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8rxz5_5db69683-41ca-4f88-a7ab-5a1174170725/kube-rbac-proxy/0.log"
Nov 03 23:32:45.089727 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:45.089689    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8rxz5_5db69683-41ca-4f88-a7ab-5a1174170725/init-textfile/0.log"
Nov 03 23:32:46.043496 ip-10-0-132-187 kubenswrapper[5647]: E1103 23:32:46.043452    5647 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lmz29" containerName="extractor"
Nov 03 23:32:47.996629 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:47.996598    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-gjlm7_01fe56b3-4ca9-416a-8480-e199577ac2d5/kube-rbac-proxy-main/0.log"
Nov 03 23:32:48.415419 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:48.415319    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-gjlm7_01fe56b3-4ca9-416a-8480-e199577ac2d5/kube-rbac-proxy-self/0.log"
Nov 03 23:32:48.828349 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:48.828317    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-gjlm7_01fe56b3-4ca9-416a-8480-e199577ac2d5/openshift-state-metrics/0.log"
Nov 03 23:32:53.430466 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:53.430421    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b759b6f89-dqvbb_9c408a46-d8c0-4c25-a708-2fbe5f78a1f9/telemeter-client/0.log"
Nov 03 23:32:53.842118 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:53.842087    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b759b6f89-dqvbb_9c408a46-d8c0-4c25-a708-2fbe5f78a1f9/reload/0.log"
Nov 03 23:32:54.254368 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:54.254337    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b759b6f89-dqvbb_9c408a46-d8c0-4c25-a708-2fbe5f78a1f9/kube-rbac-proxy/0.log"
Nov 03 23:32:54.675420 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:54.675349    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b46bc6cb-p4dg9_3ead9dc1-5785-432c-a28d-6dd002c4dc27/thanos-query/0.log"
Nov 03 23:32:55.091223 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:55.091196    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b46bc6cb-p4dg9_3ead9dc1-5785-432c-a28d-6dd002c4dc27/kube-rbac-proxy-web/0.log"
Nov 03 23:32:55.505445 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:55.505417    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b46bc6cb-p4dg9_3ead9dc1-5785-432c-a28d-6dd002c4dc27/kube-rbac-proxy/0.log"
Nov 03 23:32:55.918478 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:55.918381    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b46bc6cb-p4dg9_3ead9dc1-5785-432c-a28d-6dd002c4dc27/prom-label-proxy/0.log"
Nov 03 23:32:56.330641 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:56.330609    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b46bc6cb-p4dg9_3ead9dc1-5785-432c-a28d-6dd002c4dc27/kube-rbac-proxy-rules/0.log"
Nov 03 23:32:56.744649 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:56.744617    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b46bc6cb-p4dg9_3ead9dc1-5785-432c-a28d-6dd002c4dc27/kube-rbac-proxy-metrics/0.log"
Nov 03 23:32:57.212343 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:57.212313    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-cwdgf_2ff0e28c-db6e-4176-b124-a79a6f78ba51/discovery/0.log"
Nov 03 23:32:57.282817 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:57.282784    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:32:57.297120 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:57.297097    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d7l9z_8308da23-cdc6-4e50-91a2-9b158e17ca3b/ovn-acl-logging/0.log"
Nov 03 23:32:57.320413 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:57.320385    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-586b8ddf6d-h9m6s_82ca6b39-a72a-4632-9342-10eca3d67829/kube-auth-proxy/0.log"
Nov 03 23:32:57.744510 ip-10-0-132-187 kubenswrapper[5647]: I1103 23:32:57.744476    5647 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7f5667f86c-2z78m_171bf940-44d1-4211-9b51-4f4bc0207086/istio-proxy/0.log"